-- Logs begin at Fri 2022-06-24 02:19:11 UTC, end at Fri 2022-06-24 04:36:55 UTC. -- Jun 24 03:57:24.017352 dv-cinder-846842-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 24 03:57:24.027104 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 24 03:57:24.029161 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 24 03:57:24 2022] *** Jun 24 03:57:24.029161 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 24 03:57:24.029161 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 24 03:57:24.029161 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: nodename: dv-cinder-846842-3-iscsi Jun 24 03:57:24.029161 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: machine: x86_64 Jun 24 03:57:24.029161 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: clock source: unix Jun 24 03:57:24.029161 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: pcre jit disabled Jun 24 03:57:24.029161 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: detected number of CPU cores: 4 Jun 24 03:57:24.029161 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: current working directory: / Jun 24 03:57:24.029161 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: detected binary path: /usr/bin/uwsgi-core Jun 24 03:57:24.029161 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: your processes number limit is 47680 Jun 24 03:57:24.029874 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: your memory page size is 4096 bytes Jun 24 03:57:24.029874 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: detected max file descriptor number: 2048 Jun 24 03:57:24.029874 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: lock engine: pthread robust mutexes Jun 24 03:57:24.029874 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: thunder lock: enabled Jun 24 03:57:24.029874 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 24 03:57:24.029874 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 24 03:57:24.073425 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: Python main interpreter initialized at 0x555b3fa509c0 Jun 24 03:57:24.073425 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: python threads support enabled Jun 24 03:57:24.073425 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: your server socket listen backlog is limited to 100 connections Jun 24 03:57:24.073425 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: your mercy for graceful operations on workers is 90 seconds Jun 24 03:57:24.073841 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: mapped 403077 bytes (393 KB) for 2 cores Jun 24 03:57:24.073841 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: *** Operational MODE: preforking *** Jun 24 03:57:24.074126 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: *** uWSGI is running in multiple interpreter mode *** Jun 24 03:57:24.074126 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: spawned uWSGI master process (pid: 80586) Jun 24 03:57:24.074328 dv-cinder-846842-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 24 03:57:24.074919 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: spawned uWSGI worker 1 (pid: 80599, cores: 1) Jun 24 03:57:24.078412 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: spawned uWSGI worker 2 (pid: 80600, cores: 1) Jun 24 03:57:24.078412 dv-cinder-846842-3-iscsi devstack@keystone.service[80586]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 24 03:57:29.636616 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 03:57:29.636616 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: from cryptography.utils import int_from_bytes Jun 24 03:57:29.637462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 03:57:29.637462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: from cryptography.utils import int_from_bytes Jun 24 03:57:29.637911 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 03:57:29.637911 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: from cryptography.utils import int_from_bytes Jun 24 03:57:29.638022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 03:57:29.638022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: from cryptography.utils import int_from_bytes Jun 24 03:57:29.685920 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 03:57:29.686108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 03:57:29.686301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] command line args: [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 03:57:29.686598 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 03:57:29.686778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ================================================================================ {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 03:57:29.687032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] admin_token = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.687144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 03:57:29.687281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] config_dir = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.687373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 03:57:29.687453 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.687526 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] command line args: [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 03:57:29.687638 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] config_source = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.687778 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 03:57:29.687858 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] debug = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.687940 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ================================================================================ {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 03:57:29.688019 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.688243 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] admin_token = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.688370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.688457 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] config_dir = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.688529 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.688595 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.688763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.688877 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] config_source = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.688992 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] debug = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.689096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.689200 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.689322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.689427 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.689529 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_config_append = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.689625 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.689731 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.689731 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_dir = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.689928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.690030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_file = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.690132 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.690244 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.690330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.690490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.690567 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_config_append = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.690567 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.690743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.690869 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_dir = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.690946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.691065 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_file = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.691139 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.691262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.691344 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.691473 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.691555 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.691633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.691753 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.691836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.691922 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.692001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.692119 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.692193 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.692313 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.692966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.693099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.693319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.693548 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.693826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.693918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.694077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.694308 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.694472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.694591 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] publish_errors = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.694704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.694836 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.694959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.695073 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.695193 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.695314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.695434 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.695557 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.695679 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.695800 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.695800 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] standard_threads = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.696022 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.696149 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.696265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] publish_errors = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.696403 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.696530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.696652 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.696652 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] use_journal = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.696872 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.696992 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] use_json = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.697107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.697303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] use_stderr = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.697427 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.697542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] use_syslog = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.697656 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.697770 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.697884 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] standard_threads = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.698006 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.698122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.698237 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.698357 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.698490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.698614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.698738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.698860 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] use_journal = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.698860 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] use_json = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.699081 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.699196 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] use_stderr = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.699316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.699443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] use_syslog = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.699558 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.699678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 03:57:29.699799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.699920 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.700040 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.700164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.700279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.700394 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.700507 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.700621 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.700734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.700856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.700971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.701084 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.701204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.701325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.701441 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.701563 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.701678 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.701792 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.701914 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.702035 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.702154 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.702282 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.702426 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.702543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.702657 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.702781 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.702897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.703086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.703210 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.703325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.703325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.703546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.703666 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.703789 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.703904 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.704017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.704131 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.704244 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.704244 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.704476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.704591 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.704706 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.704819 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.704933 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.705058 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.705173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.705292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.705413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.705534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.705649 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.705774 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.705889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.706009 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.706123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.706236 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.706353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.706496 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.706597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.706699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.706795 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.706924 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.707031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.707138 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.707236 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.707339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.707437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.707534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.707657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.707784 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.707906 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.708086 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.708717 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.708840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.709046 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.709181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.709298 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.709421 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.709539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.709655 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.709655 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.709655 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.709655 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.709655 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.710188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.710303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.710454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.710570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.710702 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.710826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.710943 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.711056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.711170 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.711284 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.711410 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.711518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.711631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.711745 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.711851 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.711969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.712093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.712199 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.712312 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.712425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.712530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.712648 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.712767 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.712883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.712999 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.713122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.713239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.713353 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.713476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.713600 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.713716 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.713716 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.713931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.714046 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.714168 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.714284 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.714412 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.714528 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.714653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.714770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.714892 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.715010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.715124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.715238 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.715352 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.715467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.715653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.715779 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.715902 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.716018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.716134 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.716248 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.716385 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.716510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.716634 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.716757 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.716878 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.716994 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.717110 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.717231 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.717355 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.717472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.717586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.717701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.717816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.717930 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.718053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.718171 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.718286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.718415 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.718531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.718646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.718773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.718897 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.719013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.719127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.719242 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.719363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.719488 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.719672 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.719839 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.719959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.720075 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.720196 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.720322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.720439 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.720555 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.720670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.720785 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.720899 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.721023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.721141 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.721263 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.721381 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.721503 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.721629 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.721760 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.721884 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.722005 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.722122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.722238 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.722362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.722523 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.722638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.722753 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.722868 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.722983 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.723114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.723232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.external = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.723348 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.723471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.723604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.723725 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.723851 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.723975 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.724149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.724275 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.password = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.724391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.724515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] auth.token = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.724642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.724760 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.724875 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.724990 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.725134 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.725264 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.725264 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.725264 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.725595 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.725719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.725836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.725960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.726078 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.external = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.726194 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.726318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.726458 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.726576 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.726693 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.726810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.726926 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.727054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.password = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.727173 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.727289 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] auth.token = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.727405 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.727521 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.727638 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.727763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.727881 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.727998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.728114 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.728230 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.728352 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.728490 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.728609 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.728727 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.728727 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.728945 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.729061 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.729258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.729378 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.729500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.729618 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.729735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.729858 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.729987 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.730105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.730221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.730337 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.730480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.730597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.730730 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.730850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.730973 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.731091 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.731207 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.731322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.731446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.731565 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.731689 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.731689 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.731909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.732024 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.732148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.732266 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.732382 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.732504 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.732621 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.732736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.732861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.732979 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.733094 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.733210 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.733324 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.733446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.733571 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.733696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.733813 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.733929 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.734053 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.734178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.734296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.734426 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.734543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.734658 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.734773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.734901 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.735027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.735144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.735260 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.735375 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.735491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.735614 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.735731 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.735853 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.735971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.736087 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.736208 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.736336 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.736454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.736569 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.736685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.736806 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.736923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.737048 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.737173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.737296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.737413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.737528 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.737634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.737757 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.737874 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.737990 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.738106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.738221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.738335 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.738553 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.738674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.738790 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.738905 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.739027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.739145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.739288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.739408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.739530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.739648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.739764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.739879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.740004 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.740121 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.740239 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.740355 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.740479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.740596 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.740720 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.740837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.740953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.741068 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.741184 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.741184 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.741184 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.741513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.741629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.741745 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.741860 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.741981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.742111 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.742228 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.742345 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.742478 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.742594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.742709 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.742838 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.743205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.743324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.743439 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.743555 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.743670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.743794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.743911 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.744026 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.744141 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.744256 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.744372 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.744495 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.744612 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.744727 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.744843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.744958 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.745081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.745198 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.745313 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.745429 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.745545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.745660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.745787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.745904 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.746020 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.746135 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.746252 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.746424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.746556 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.746680 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.746797 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.746913 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.747028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.747143 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.747266 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.747384 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.747500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.747615 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.747730 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.747845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.747969 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.748086 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.748201 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.748316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.748431 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.748547 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.748671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.748788 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.748903 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.749018 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.749133 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.749248 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.749371 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.749488 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.749604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.749718 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.749833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.749947 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.750071 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.750187 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.750303 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.750430 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.750546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.750663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.750790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.750790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.751008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.751124 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.751239 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.751354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.751478 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.751595 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.751711 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.751825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.751940 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.752055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.752242 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.752361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.752476 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.752592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.752708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.752824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.752948 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.753065 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.753181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.753296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.753411 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.753527 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.753651 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.753768 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.753884 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.753999 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.754114 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.754229 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.754352 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.754499 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.754615 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.754732 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.755359 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.755540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.755677 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.755797 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.755915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.756032 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.756148 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.756264 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.756392 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.756510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.756626 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.756742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.756857 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.756974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.757104 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] role.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.757221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.757337 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] role.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.757453 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.757568 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.757684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.757808 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.757808 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.757808 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.758130 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.758245 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.758361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.758523 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.758642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.758758 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.758873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.758989 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.759114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.759232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.759348 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.759463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.759579 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.759695 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.759819 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.759936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.760060 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.760177 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.760293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.760408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.760532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] role.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.760649 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.760772 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] role.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.760772 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.760990 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.761106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.761231 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.761349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.761456 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.761572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.761687 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.761803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.761927 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.761927 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.762146 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.762146 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.762364 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.762501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.762631 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.762631 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.762851 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.762967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.763082 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.763198 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.763324 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.763441 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.763564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.763681 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.763797 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.763913 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.764037 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.764154 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.764270 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.764386 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.764502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.764617 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.764741 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.764859 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.764975 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.765090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.765205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.765329 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.765447 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.765562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.765678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.765794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.765909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.766034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.766151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.766267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.766404 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.766522 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.766637 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.766765 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.766884 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.767000 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.767115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.767231 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.767346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.767472 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.767589 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.767705 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.767820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.767936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.768051 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.768175 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.768292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.768408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.768524 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.768639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.768755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.768879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.768996 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.769111 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.769262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.769380 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.connection = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.769513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.769646 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.769768 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.769894 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.770011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.770127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.770242 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.770358 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.770594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.771209 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.771327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.771476 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.771628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.771746 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.771872 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.771990 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.772106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.772222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.772339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.772454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.772578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.772697 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.772812 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.772928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.773043 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.773158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.773283 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.773400 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.773400 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.773400 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.773719 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.773835 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.774026 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.connection = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.774155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.774284 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.774420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80600) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 03:57:29.774538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.774538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.774538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.774538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.774538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.774538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.774538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.774538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.774538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.775517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.775517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.775517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.775517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.775517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.775517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.776148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.776148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 03:57:29.776148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80599) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 03:57:29.803142 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:29.803372 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:29.803597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:29.807792 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:29.808024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:29.808248 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:29.812288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:29.816674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:29.841973 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:29.842203 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:29.844871 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:29.845104 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:29.984783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:29.995027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.035500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.035731 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.042104 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.042104 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.100621 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.100840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.101060 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.101276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.101490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.101702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.101983 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.102213 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.102461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.102681 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.102896 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.103110 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.127178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.127408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.127633 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.127856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.128073 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.128289 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.128507 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.129138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.129374 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.129602 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.129848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.130500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.130728 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.130946 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.139704 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.141018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.152756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.152987 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.153215 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.153432 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.153649 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.153860 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.153944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.154070 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80600) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.154161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.154293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.154537 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.154752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.154962 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.155173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80599) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 03:57:30.171631 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.171856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 03:57:30.172170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.172253 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.172370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 03:57:30.172453 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 03:57:30.172636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.172719 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.172892 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.172974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 03:57:30.173145 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.173237 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.173396 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.173478 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.173652 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.173905 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.184346 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.184618 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.184884 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.185164 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.185246 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.185476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.185741 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.186018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.193909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.195219 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.197180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.197450 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.197722 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.197995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.198313 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.198594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.198858 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.199125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.199389 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.199656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.199913 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.203015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.203300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.203592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.203877 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.204160 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.204421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.204692 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.204966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.205234 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.205502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.205765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.226851 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.228450 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.228724 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.233974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.234364 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.234663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.237050 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.238661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.238937 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.242123 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.242419 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.244937 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.245329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.245609 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.247130 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.247413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.247691 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.247966 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.248233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.248499 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.248761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.249023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.252878 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.253158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.258681 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.258959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.259232 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.259503 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.259771 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.260036 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.260300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.260561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.268188 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.268461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.273666 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.277295 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.277688 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.280131 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.280406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.282695 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.284976 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.286364 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.289347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.289764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.290110 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.290426 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.290708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.290982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.291249 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.291517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.294800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.298384 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.302473 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.302770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.303053 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.303338 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.303615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.303892 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.310620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.310899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.311158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.311420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.319972 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.321669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.321938 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.325251 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.325545 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.325842 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.326147 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.326502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.327451 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.327731 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.327996 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.328273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.336691 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.338394 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.338665 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.339736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.340011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.340292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.340568 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.341838 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.342120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.342203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.342415 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.342501 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.342697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.342781 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.342977 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.357249 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.357525 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.357817 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.358095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.359691 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.359983 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.360274 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.361065 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.364985 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.368722 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.370479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.372100 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.372373 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.372648 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.378961 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.380004 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.381594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.382806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.384831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.385119 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.385503 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.385777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.386085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.387831 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.389471 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.389748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.390023 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.394839 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.395121 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.395395 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.397731 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.399463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.402768 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.403062 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.403319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80600) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.403450 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.403594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.403723 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.403865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.404131 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.404392 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.404765 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.405038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.405308 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.405584 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.405855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.406120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.406396 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.406658 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.406927 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80600) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.413761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.414040 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.414316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.422110 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80599) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 03:57:30.422412 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.422689 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.422959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.423226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.423592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.423875 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.424140 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.424416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.424691 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.424957 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.425214 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.425473 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.425743 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80599) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 03:57:30.445974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555b3fa509c0 pid: 80600 (default app) Jun 24 03:57:30.449422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdc15f-d670-4391-9e21-7e887e169db7 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:30.449714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdc15f-d670-4391-9e21-7e887e169db7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:30.450003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdc15f-d670-4391-9e21-7e887e169db7 None None] PATH_INFO: `/v3` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:30.454166 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Fri Jun 24 03:57:30 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:57:30.470885 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555b3fa509c0 pid: 80599 (default app) Jun 24 03:57:39.207901 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeff075-9f00-402e-8565-8525dc400f3c None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:39.208429 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeff075-9f00-402e-8565-8525dc400f3c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:39.208429 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeff075-9f00-402e-8565-8525dc400f3c None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:39.210905 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 2/2] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:57:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:57:39.219135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158f3d8-e044-442e-89bb-b7dc1ab88fff None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:39.219551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158f3d8-e044-442e-89bb-b7dc1ab88fff None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:39.219702 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2158f3d8-e044-442e-89bb-b7dc1ab88fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:39.290129 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: INFO dbcounter [None req-2158f3d8-e044-442e-89bb-b7dc1ab88fff None None] Registered counter for database keystone Jun 24 03:57:39.311234 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG oslo_db.sqlalchemy.engines [None req-2158f3d8-e044-442e-89bb-b7dc1ab88fff None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80599) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 03:57:39.317356 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writer thread running {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 03:57:39.497658 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG passlib.handlers.bcrypt [None req-2158f3d8-e044-442e-89bb-b7dc1ab88fff None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80599) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 03:57:39.498081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG passlib.handlers.bcrypt [None req-2158f3d8-e044-442e-89bb-b7dc1ab88fff None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80599) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 03:57:39.553029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2158f3d8-e044-442e-89bb-b7dc1ab88fff None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:57:39.554664 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2158f3d8-e044-442e-89bb-b7dc1ab88fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:57:39.674824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1/3] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:57:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 24 03:57:39.847998 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeff075-9f00-402e-8565-8525dc400f3c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:39.919188 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: INFO dbcounter [None req-6eeff075-9f00-402e-8565-8525dc400f3c None None] Registered counter for database keystone Jun 24 03:57:39.939120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG oslo_db.sqlalchemy.engines [None req-6eeff075-9f00-402e-8565-8525dc400f3c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80600) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 03:57:39.945543 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writer thread running {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 03:57:40.077732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeff075-9f00-402e-8565-8525dc400f3c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:40.081992 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc8c5a-0d14-48f8-a206-1dc1716b7722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:40.082698 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc8c5a-0d14-48f8-a206-1dc1716b7722 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:40.082988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc8c5a-0d14-48f8-a206-1dc1716b7722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:40.083300 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc8c5a-0d14-48f8-a206-1dc1716b7722 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:40.091588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc8c5a-0d14-48f8-a206-1dc1716b7722 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:40.144506 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: WARNING keystone.common.rbac_enforcer.enforcer [None req-eecc8c5a-0d14-48f8-a206-1dc1716b7722 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 03:57:40.144850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc8c5a-0d14-48f8-a206-1dc1716b7722 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:40.154687 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 3/4] 10.222.0.16 () {58 vars in 1324 bytes} [Fri Jun 24 03:57:39 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 03:57:40.160828 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158f3d8-e044-442e-89bb-b7dc1ab88fff None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:40.175892 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2158f3d8-e044-442e-89bb-b7dc1ab88fff None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:40.180431 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d5198-c17b-4f41-a2ac-293fe366a509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:40.181533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d5198-c17b-4f41-a2ac-293fe366a509 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:40.181825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d5198-c17b-4f41-a2ac-293fe366a509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:40.182105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d5198-c17b-4f41-a2ac-293fe366a509 admin admin] PATH_INFO: `/v3/projects/6a1c0f0b20b844d7b8311813151d93aa` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:40.186160 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d5198-c17b-4f41-a2ac-293fe366a509 admin admin] RBAC: Authorizing `identity:get_project(project_id=6a1c0f0b20b844d7b8311813151d93aa)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:40.237501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: WARNING keystone.common.rbac_enforcer.enforcer [None req-918d5198-c17b-4f41-a2ac-293fe366a509 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 03:57:40.238052 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d5198-c17b-4f41-a2ac-293fe366a509 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:40.242546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 2/5] 10.222.0.16 () {56 vars in 1207 bytes} [Fri Jun 24 03:57:40 2022] GET /identity/v3/projects/6a1c0f0b20b844d7b8311813151d93aa => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:57:42.250663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87570467-7496-4cab-8732-febdde93d641 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:42.251166 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87570467-7496-4cab-8732-febdde93d641 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:42.251249 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87570467-7496-4cab-8732-febdde93d641 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:42.256429 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 4/6] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:57:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:57:42.261741 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2546672-1ede-4e2a-ba2f-2aa0011b0abc None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:42.262166 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2546672-1ede-4e2a-ba2f-2aa0011b0abc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:42.262312 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2546672-1ede-4e2a-ba2f-2aa0011b0abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:42.281366 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c2546672-1ede-4e2a-ba2f-2aa0011b0abc None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:57:42.282941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c2546672-1ede-4e2a-ba2f-2aa0011b0abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:57:42.291045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 3/7] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:57:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 24 03:57:42.312822 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87570467-7496-4cab-8732-febdde93d641 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:42.325462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87570467-7496-4cab-8732-febdde93d641 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:42.329689 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c75fb0-c9fd-4411-94a2-b99c2a0cfab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:42.330324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c75fb0-c9fd-4411-94a2-b99c2a0cfab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:42.330643 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c75fb0-c9fd-4411-94a2-b99c2a0cfab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:42.330927 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c75fb0-c9fd-4411-94a2-b99c2a0cfab9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:42.339505 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c75fb0-c9fd-4411-94a2-b99c2a0cfab9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:42.341272 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c75fb0-c9fd-4411-94a2-b99c2a0cfab9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:42.350069 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 5/8] 10.222.0.16 () {58 vars in 1324 bytes} [Fri Jun 24 03:57:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 03:57:42.355393 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2546672-1ede-4e2a-ba2f-2aa0011b0abc None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:42.364345 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2546672-1ede-4e2a-ba2f-2aa0011b0abc None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:42.369027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b91be6-a1a3-4e08-ac70-c24188f5b9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:42.369888 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b91be6-a1a3-4e08-ac70-c24188f5b9c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:42.370177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b91be6-a1a3-4e08-ac70-c24188f5b9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:42.370495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b91be6-a1a3-4e08-ac70-c24188f5b9c1 admin admin] PATH_INFO: `/v3/users/c531d2732de94f47a66f16db7047917f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:42.371882 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b91be6-a1a3-4e08-ac70-c24188f5b9c1 admin admin] RBAC: Authorizing `identity:get_user(user_id=c531d2732de94f47a66f16db7047917f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:42.373720 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b91be6-a1a3-4e08-ac70-c24188f5b9c1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:42.375319 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 4/9] 10.222.0.16 () {56 vars in 1198 bytes} [Fri Jun 24 03:57:42 2022] GET /identity/v3/users/c531d2732de94f47a66f16db7047917f => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:57:44.472484 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b84da-4c18-4a50-9ac6-64c4445ae9cc None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:44.473005 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b84da-4c18-4a50-9ac6-64c4445ae9cc None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:44.473089 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b84da-4c18-4a50-9ac6-64c4445ae9cc None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:44.474961 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 6/10] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:57:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:57:44.480166 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa669d2d-fdcf-407e-b796-f99fc21caff4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:44.480563 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa669d2d-fdcf-407e-b796-f99fc21caff4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:44.480740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa669d2d-fdcf-407e-b796-f99fc21caff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:44.498113 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-aa669d2d-fdcf-407e-b796-f99fc21caff4 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:57:44.499345 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-aa669d2d-fdcf-407e-b796-f99fc21caff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:57:44.506162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5103a-8d1a-47b9-8901-e2c99db205f8 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:44.506485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5103a-8d1a-47b9-8901-e2c99db205f8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:44.506771 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad5103a-8d1a-47b9-8901-e2c99db205f8 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:44.508110 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 7/11] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:57:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 03:57:44.513158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd461a8-3d1a-462a-b8f1-7988cf2409a6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:44.513443 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd461a8-3d1a-462a-b8f1-7988cf2409a6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:44.513718 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd461a8-3d1a-462a-b8f1-7988cf2409a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:44.554614 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG passlib.handlers.bcrypt [None req-9dd461a8-3d1a-462a-b8f1-7988cf2409a6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80600) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 03:57:44.555037 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG passlib.handlers.bcrypt [None req-9dd461a8-3d1a-462a-b8f1-7988cf2409a6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80600) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 03:57:44.581221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 5/12] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:57:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:57:44.601567 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9dd461a8-3d1a-462a-b8f1-7988cf2409a6 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:57:44.603190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9dd461a8-3d1a-462a-b8f1-7988cf2409a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:57:44.604628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa669d2d-fdcf-407e-b796-f99fc21caff4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:44.628739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:44.703658 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 8/13] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:57:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:57:44.709418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa669d2d-fdcf-407e-b796-f99fc21caff4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:44.727412 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd461a8-3d1a-462a-b8f1-7988cf2409a6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:44.743468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b7dce-726a-451d-a531-3cb01ac4eeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:44.744279 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b7dce-726a-451d-a531-3cb01ac4eeba None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:44.744568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b7dce-726a-451d-a531-3cb01ac4eeba None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:44.744852 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b7dce-726a-451d-a531-3cb01ac4eeba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:44.756064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b7dce-726a-451d-a531-3cb01ac4eeba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:57:44.767310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-2f9b7dce-726a-451d-a531-3cb01ac4eeba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:57:44.767310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:57:44.767310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:57:44.767310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:57:44.767310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:57:44.767310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:57:44.767310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:57:44.767310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:57:44.767310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:57:44.767310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:57:44.767310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:57:44.767310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:57:44.767310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:57:44.767310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:57:44.767310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:57:44.767310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:57:44.768164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:57:44.768164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:57:44.768164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:57:44.768164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:57:44.768164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:57:44.768164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:57:44.768164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:57:44.768164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:57:44.768164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:57:44.768164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:57:44.768164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:57:44.768164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:57:44.768164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:57:44.768164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:57:44.768164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:57:44.768164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:57:44.769013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:57:44.769013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:57:44.769013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:57:44.769013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:57:44.769013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:57:44.769013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:57:44.769013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:57:44.769013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:57:44.769013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:57:44.769013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:57:44.769013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:57:44.769013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:57:44.769013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 6/14] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:57:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:57:44.774990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b7dce-726a-451d-a531-3cb01ac4eeba None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:44.828322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd461a8-3d1a-462a-b8f1-7988cf2409a6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:44.830582 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:44.857088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:44.866084 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d6802-fbd1-4a4d-96fc-b4e12c27b708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:44.867021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d6802-fbd1-4a4d-96fc-b4e12c27b708 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:44.867313 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d6802-fbd1-4a4d-96fc-b4e12c27b708 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:44.867596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d6802-fbd1-4a4d-96fc-b4e12c27b708 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:44.876009 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b7dce-726a-451d-a531-3cb01ac4eeba None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:44.913023 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acba253-cf9d-430d-a1a8-3b6e2f80c56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:44.913690 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acba253-cf9d-430d-a1a8-3b6e2f80c56b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:44.913975 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acba253-cf9d-430d-a1a8-3b6e2f80c56b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:44.914255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acba253-cf9d-430d-a1a8-3b6e2f80c56b None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:44.915051 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acba253-cf9d-430d-a1a8-3b6e2f80c56b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:57:44.915358 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acba253-cf9d-430d-a1a8-3b6e2f80c56b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:44.916757 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acba253-cf9d-430d-a1a8-3b6e2f80c56b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:44.928530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 7/15] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:57:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:57:44.934244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acba253-cf9d-430d-a1a8-3b6e2f80c56b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:44.940440 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d6802-fbd1-4a4d-96fc-b4e12c27b708 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:44.941970 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d6802-fbd1-4a4d-96fc-b4e12c27b708 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:45.033218 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acba253-cf9d-430d-a1a8-3b6e2f80c56b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:45.042931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 9/16] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:57:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 03:57:45.048490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d6802-fbd1-4a4d-96fc-b4e12c27b708 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:45.051346 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:45.071412 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359cbca-354d-4cb8-8ebb-3913de925309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:45.072297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359cbca-354d-4cb8-8ebb-3913de925309 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:45.072584 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359cbca-354d-4cb8-8ebb-3913de925309 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:45.072863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359cbca-354d-4cb8-8ebb-3913de925309 None admin] PATH_INFO: `/v3/users/c531d2732de94f47a66f16db7047917f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:45.074085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359cbca-354d-4cb8-8ebb-3913de925309 None admin] RBAC: Authorizing `identity:get_user(user_id=c531d2732de94f47a66f16db7047917f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:45.075619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359cbca-354d-4cb8-8ebb-3913de925309 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:45.077426 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 8/17] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:57:44 2022] GET /identity/v3/users/c531d2732de94f47a66f16db7047917f => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:57:45.082641 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359cbca-354d-4cb8-8ebb-3913de925309 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:45.097260 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:45.148261 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d6802-fbd1-4a4d-96fc-b4e12c27b708 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:45.183555 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc44518-833d-4882-9847-8146136e878a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:45.184350 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc44518-833d-4882-9847-8146136e878a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:45.184638 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc44518-833d-4882-9847-8146136e878a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:45.184865 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359cbca-354d-4cb8-8ebb-3913de925309 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:45.184960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc44518-833d-4882-9847-8146136e878a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:45.203565 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc44518-833d-4882-9847-8146136e878a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:45.205057 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc44518-833d-4882-9847-8146136e878a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:45.218413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013e25e-fd06-4394-8e15-07e73b1861ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:45.219210 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013e25e-fd06-4394-8e15-07e73b1861ea None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:45.219493 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013e25e-fd06-4394-8e15-07e73b1861ea None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:45.219769 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a013e25e-fd06-4394-8e15-07e73b1861ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:45.220954 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013e25e-fd06-4394-8e15-07e73b1861ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:45.222412 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a013e25e-fd06-4394-8e15-07e73b1861ea None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:45.224133 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 9/18] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:57:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:57:45.228427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-bbc44518-833d-4882-9847-8146136e878a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:57:45.228427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:57:45.228427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:57:45.228427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:57:45.228427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:57:45.228427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:57:45.228427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:57:45.228427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:57:45.228427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:57:45.228427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:57:45.228427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:57:45.228427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:57:45.228427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:57:45.228427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:57:45.228427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:57:45.229255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:57:45.229255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:57:45.229255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:57:45.229255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:57:45.229255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:57:45.229255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:57:45.229255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:57:45.229255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:57:45.229255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:57:45.229255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:57:45.229255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:57:45.229255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:57:45.229255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:57:45.229255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:57:45.229255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:57:45.229255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:57:45.229255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:57:45.230063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:57:45.230063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:57:45.230063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:57:45.230063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:57:45.230063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 10/19] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:57:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:57:45.230316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013e25e-fd06-4394-8e15-07e73b1861ea None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:45.235103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc44518-833d-4882-9847-8146136e878a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:45.291342 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:45.308888 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:45.328112 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a013e25e-fd06-4394-8e15-07e73b1861ea None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:45.334734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc44518-833d-4882-9847-8146136e878a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:45.362587 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e478c-a62b-4c0d-ab45-dc4715ef51a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:45.363252 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e478c-a62b-4c0d-ab45-dc4715ef51a9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:45.363540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e478c-a62b-4c0d-ab45-dc4715ef51a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:45.363823 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e478c-a62b-4c0d-ab45-dc4715ef51a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:45.364812 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4e478c-a62b-4c0d-ab45-dc4715ef51a9 None admin] RBAC: Adding query filter params (role.id=f0f7e28cbe314921afdfb657b163436f, scope.domain.id=default, user.id=c531d2732de94f47a66f16db7047917f) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:57:45.365130 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4e478c-a62b-4c0d-ab45-dc4715ef51a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:45.366623 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4e478c-a62b-4c0d-ab45-dc4715ef51a9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:45.367538 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ada05-2eab-4e43-880b-a9a687111e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:45.368455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ada05-2eab-4e43-880b-a9a687111e38 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:45.368739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ada05-2eab-4e43-880b-a9a687111e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:45.369016 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ada05-2eab-4e43-880b-a9a687111e38 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:45.369737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ada05-2eab-4e43-880b-a9a687111e38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:57:45.370045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ada05-2eab-4e43-880b-a9a687111e38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:45.371455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ada05-2eab-4e43-880b-a9a687111e38 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:45.373762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 10/20] 10.222.0.16 () {56 vars in 1322 bytes} [Fri Jun 24 03:57:45 2022] GET /identity/v3/role_assignments?user.id=c531d2732de94f47a66f16db7047917f&scope.domain.id=default&role.id=f0f7e28cbe314921afdfb657b163436f => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:57:45.390817 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 11/21] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:57:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:57:47.628280 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9499a-0cd1-4952-8669-5d2361e5eb0e None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:47.628799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9499a-0cd1-4952-8669-5d2361e5eb0e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:47.628888 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9499a-0cd1-4952-8669-5d2361e5eb0e None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:47.630788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 11/22] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:57:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:57:47.652193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35687d72-0267-4cc8-9d0b-24b9865c46eb None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:47.652682 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35687d72-0267-4cc8-9d0b-24b9865c46eb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:47.652753 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35687d72-0267-4cc8-9d0b-24b9865c46eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:47.679384 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-35687d72-0267-4cc8-9d0b-24b9865c46eb None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:57:47.680590 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-35687d72-0267-4cc8-9d0b-24b9865c46eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:57:47.775960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 12/23] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:57:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:57:47.798593 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9499a-0cd1-4952-8669-5d2361e5eb0e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:47.924205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800ea85-2bd4-4fb4-9984-776a9c0ec327 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:47.924527 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800ea85-2bd4-4fb4-9984-776a9c0ec327 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:47.924702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800ea85-2bd4-4fb4-9984-776a9c0ec327 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:47.926062 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 13/24] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 03:57:47.926396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9499a-0cd1-4952-8669-5d2361e5eb0e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:47.941556 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8733f52-7ab4-433b-a370-535d52bac551 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:47.941841 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8733f52-7ab4-433b-a370-535d52bac551 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:47.942117 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8733f52-7ab4-433b-a370-535d52bac551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:47.964058 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b8733f52-7ab4-433b-a370-535d52bac551 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:57:47.965249 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b8733f52-7ab4-433b-a370-535d52bac551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:57:48.000275 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1a8b7-383b-4f81-a558-39f8cec1aad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:48.001077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1a8b7-383b-4f81-a558-39f8cec1aad9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:48.001362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1a8b7-383b-4f81-a558-39f8cec1aad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:48.001639 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1a8b7-383b-4f81-a558-39f8cec1aad9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:48.021532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1a8b7-383b-4f81-a558-39f8cec1aad9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:57:48.029302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-63b1a8b7-383b-4f81-a558-39f8cec1aad9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:57:48.029302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:57:48.029302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:57:48.029302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:57:48.029302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:57:48.029302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:57:48.029302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:57:48.029302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:57:48.029302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:57:48.029302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:57:48.029302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:57:48.029302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:57:48.029302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:57:48.029302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:57:48.029302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:57:48.029302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:57:48.030192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:57:48.030192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:57:48.030192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:57:48.030192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:57:48.030192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:57:48.030192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:57:48.030192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:57:48.030192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:57:48.030192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:57:48.030192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:57:48.030192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:57:48.030192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:57:48.030192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:57:48.030192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:57:48.030192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:57:48.030192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:57:48.030967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:57:48.030967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:57:48.030967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:57:48.030967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:57:48.030967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:57:48.030967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:57:48.030967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:57:48.030967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:57:48.030967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:57:48.030967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:57:48.030967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:57:48.030967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:57:48.033077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 12/25] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:57:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:57:48.039896 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1a8b7-383b-4f81-a558-39f8cec1aad9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:48.048269 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:48.074712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:48.128763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 14/26] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:57:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:57:48.130319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc63df-f9ca-462c-b6ea-b53be0673e0c None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:48.130734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc63df-f9ca-462c-b6ea-b53be0673e0c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:48.130914 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc63df-f9ca-462c-b6ea-b53be0673e0c None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:48.132515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 15/27] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:57:48.137731 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88397ff-c6b2-4628-8303-56e7f7729d47 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:48.138031 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88397ff-c6b2-4628-8303-56e7f7729d47 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:48.138322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88397ff-c6b2-4628-8303-56e7f7729d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:48.179534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b88397ff-c6b2-4628-8303-56e7f7729d47 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:57:48.180685 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b88397ff-c6b2-4628-8303-56e7f7729d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:57:48.195159 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1a8b7-383b-4f81-a558-39f8cec1aad9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:48.241291 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac3b45-a046-4dd6-8de1-3228cc8ae30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:48.241974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac3b45-a046-4dd6-8de1-3228cc8ae30b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:48.242266 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac3b45-a046-4dd6-8de1-3228cc8ae30b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:48.242575 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac3b45-a046-4dd6-8de1-3228cc8ae30b None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:48.243603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac3b45-a046-4dd6-8de1-3228cc8ae30b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:57:48.243917 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac3b45-a046-4dd6-8de1-3228cc8ae30b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:48.245327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac3b45-a046-4dd6-8de1-3228cc8ae30b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:48.270189 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 13/28] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:57:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:57:48.271323 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac3b45-a046-4dd6-8de1-3228cc8ae30b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:48.306222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 16/29] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:57:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:57:48.307104 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88397ff-c6b2-4628-8303-56e7f7729d47 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:48.383005 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:48.421353 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88397ff-c6b2-4628-8303-56e7f7729d47 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:48.422715 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac3b45-a046-4dd6-8de1-3228cc8ae30b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:48.429531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:48.456908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b1d64-5d88-4972-a4bd-85816fe87d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:48.457717 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b1d64-5d88-4972-a4bd-85816fe87d73 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:48.458030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b1d64-5d88-4972-a4bd-85816fe87d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:48.458311 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b1d64-5d88-4972-a4bd-85816fe87d73 None admin] PATH_INFO: `/v3/users/c531d2732de94f47a66f16db7047917f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:48.459906 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b1d64-5d88-4972-a4bd-85816fe87d73 None admin] RBAC: Authorizing `identity:get_user(user_id=c531d2732de94f47a66f16db7047917f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:48.461379 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b1d64-5d88-4972-a4bd-85816fe87d73 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:48.462309 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3a427-1abc-4495-a1e4-c97db2db79e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:48.463155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 17/30] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:57:48 2022] GET /identity/v3/users/c531d2732de94f47a66f16db7047917f => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:57:48.463263 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3a427-1abc-4495-a1e4-c97db2db79e5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:48.463586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3a427-1abc-4495-a1e4-c97db2db79e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:48.463947 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f3a427-1abc-4495-a1e4-c97db2db79e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:48.464333 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b24355-d456-440f-ab87-8339bc268b09 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:48.464624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b24355-d456-440f-ab87-8339bc268b09 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:48.464904 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b24355-d456-440f-ab87-8339bc268b09 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:48.466125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 18/31] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:57:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:57:48.466847 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b24355-d456-440f-ab87-8339bc268b09 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:48.480478 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3a427-1abc-4495-a1e4-c97db2db79e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:48.481991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f3a427-1abc-4495-a1e4-c97db2db79e5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:48.501701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-d0f3a427-1abc-4495-a1e4-c97db2db79e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:57:48.501701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:57:48.501701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:57:48.501701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:57:48.501701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:57:48.501701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:57:48.501701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:57:48.501701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:57:48.501701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:57:48.501701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:57:48.501701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:57:48.501701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:57:48.501701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:57:48.501701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:57:48.501701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:57:48.502555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:57:48.502555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:57:48.502555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:57:48.502555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:57:48.502555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:57:48.502555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:57:48.502555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:57:48.502555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:57:48.502555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:57:48.502555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:57:48.502555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:57:48.502555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:57:48.502555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:57:48.502555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:57:48.502555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:57:48.502555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:57:48.502555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:57:48.504151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:57:48.504151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:57:48.504151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:57:48.504151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:57:48.504151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 14/32] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:57:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:57:48.504151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3a427-1abc-4495-a1e4-c97db2db79e5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:48.586571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b24355-d456-440f-ab87-8339bc268b09 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:48.622895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98d700-5153-4d7e-8839-0c9a28944b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:48.623632 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98d700-5153-4d7e-8839-0c9a28944b47 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:48.623921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98d700-5153-4d7e-8839-0c9a28944b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:48.624202 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee98d700-5153-4d7e-8839-0c9a28944b47 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:48.625017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee98d700-5153-4d7e-8839-0c9a28944b47 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:48.626607 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee98d700-5153-4d7e-8839-0c9a28944b47 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:48.635195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f3a427-1abc-4495-a1e4-c97db2db79e5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:48.653991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:48.671384 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335a1b6-feca-46fe-9fed-170c72a95ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:48.672185 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335a1b6-feca-46fe-9fed-170c72a95ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:48.672478 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335a1b6-feca-46fe-9fed-170c72a95ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:48.672758 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335a1b6-feca-46fe-9fed-170c72a95ba7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:48.673958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335a1b6-feca-46fe-9fed-170c72a95ba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:48.675470 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335a1b6-feca-46fe-9fed-170c72a95ba7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:48.677328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 15/33] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:57:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:57:48.682782 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e4a20-262f-4a97-9d0a-8e836f46c556 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:48.683077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e4a20-262f-4a97-9d0a-8e836f46c556 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:48.683362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5e4a20-262f-4a97-9d0a-8e836f46c556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:48.699634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5e5e4a20-262f-4a97-9d0a-8e836f46c556 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:57:48.700843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5e5e4a20-262f-4a97-9d0a-8e836f46c556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:57:48.767030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 16/34] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:57:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:57:48.767623 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e4a20-262f-4a97-9d0a-8e836f46c556 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:48.888711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5e4a20-262f-4a97-9d0a-8e836f46c556 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:48.905048 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 19/35] 10.222.0.16 () {60 vars in 1131 bytes} [Fri Jun 24 03:57:48 2022] POST /identity/v3/roles => generated 225 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:57:48.906667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98d700-5153-4d7e-8839-0c9a28944b47 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:48.907280 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:48.927644 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e27cd8-ce41-459e-8ed4-cc4ccfbf79a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:48.928339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e27cd8-ce41-459e-8ed4-cc4ccfbf79a4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:48.928630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e27cd8-ce41-459e-8ed4-cc4ccfbf79a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:48.928916 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e27cd8-ce41-459e-8ed4-cc4ccfbf79a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:48.929801 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e27cd8-ce41-459e-8ed4-cc4ccfbf79a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:57:48.930066 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e27cd8-ce41-459e-8ed4-cc4ccfbf79a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:48.931586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e27cd8-ce41-459e-8ed4-cc4ccfbf79a4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:48.951747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:48.966825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 17/36] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:57:48.972484 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e27cd8-ce41-459e-8ed4-cc4ccfbf79a4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:49.039176 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee98d700-5153-4d7e-8839-0c9a28944b47 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:49.083675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9b6a7-790c-4774-9654-d3141f1828d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:49.084684 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9b6a7-790c-4774-9654-d3141f1828d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:49.085314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9b6a7-790c-4774-9654-d3141f1828d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:49.085716 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9b6a7-790c-4774-9654-d3141f1828d9 None admin] PATH_INFO: `/v3/domains/default/users/c531d2732de94f47a66f16db7047917f/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:49.095327 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9b6a7-790c-4774-9654-d3141f1828d9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c531d2732de94f47a66f16db7047917f, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:49.097002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9b6a7-790c-4774-9654-d3141f1828d9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:49.142014 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e27cd8-ce41-459e-8ed4-cc4ccfbf79a4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:49.184925 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486ce73-7529-4f56-bafc-e42c3ec71db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:49.185677 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486ce73-7529-4f56-bafc-e42c3ec71db0 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:49.186075 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486ce73-7529-4f56-bafc-e42c3ec71db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:49.186421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486ce73-7529-4f56-bafc-e42c3ec71db0 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:49.187202 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e486ce73-7529-4f56-bafc-e42c3ec71db0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:49.188753 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e486ce73-7529-4f56-bafc-e42c3ec71db0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:49.220738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 20/37] 10.222.0.16 () {58 vars in 1361 bytes} [Fri Jun 24 03:57:48 2022] PUT /identity/v3/domains/default/users/c531d2732de94f47a66f16db7047917f/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:57:49.221368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9b6a7-790c-4774-9654-d3141f1828d9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:49.350939 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:49.360913 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9b6a7-790c-4774-9654-d3141f1828d9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:49.414515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aaf5cb-f94a-43db-877e-5818b8772bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:49.415496 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aaf5cb-f94a-43db-877e-5818b8772bb7 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:49.415931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aaf5cb-f94a-43db-877e-5818b8772bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:49.416381 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aaf5cb-f94a-43db-877e-5818b8772bb7 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:49.417279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aaf5cb-f94a-43db-877e-5818b8772bb7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:49.419243 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aaf5cb-f94a-43db-877e-5818b8772bb7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:49.505941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 18/38] 10.222.0.16 () {60 vars in 1131 bytes} [Fri Jun 24 03:57:48 2022] POST /identity/v3/roles => generated 219 bytes in 539 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:57:49.507518 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d822f30-4d10-4ccf-8232-50954d614976 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:49.507812 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d822f30-4d10-4ccf-8232-50954d614976 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:49.508099 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d822f30-4d10-4ccf-8232-50954d614976 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:49.509372 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 19/39] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:57:49.515207 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf876fa-e757-42a8-a328-201512b36b26 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:49.515504 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf876fa-e757-42a8-a328-201512b36b26 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:49.515795 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf876fa-e757-42a8-a328-201512b36b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:49.535954 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8cf876fa-e757-42a8-a328-201512b36b26 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:57:49.538164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8cf876fa-e757-42a8-a328-201512b36b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:57:49.550802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 21/40] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jun 24 03:57:49 2022] POST /identity/v3/projects => generated 301 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:57:49.575354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:49.621494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730827d-db25-4b95-9b8b-9553652822d5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:49.621809 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730827d-db25-4b95-9b8b-9553652822d5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:49.622097 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730827d-db25-4b95-9b8b-9553652822d5 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:49.623176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 20/41] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:57:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:57:49.623609 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 22/42] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 03:57:49.629386 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772ac8c-8d8f-4b46-ba95-e1fbce976266 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:49.630306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772ac8c-8d8f-4b46-ba95-e1fbce976266 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:49.630622 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c772ac8c-8d8f-4b46-ba95-e1fbce976266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:49.662891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c772ac8c-8d8f-4b46-ba95-e1fbce976266 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:57:49.664240 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c772ac8c-8d8f-4b46-ba95-e1fbce976266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:57:49.689735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730827d-db25-4b95-9b8b-9553652822d5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:49.782318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 21/43] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:57:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:57:49.806868 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772ac8c-8d8f-4b46-ba95-e1fbce976266 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:49.860227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730827d-db25-4b95-9b8b-9553652822d5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:49.905101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14884c98-c60f-4223-89ba-d20ebc389fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:49.905873 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14884c98-c60f-4223-89ba-d20ebc389fb5 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:49.906171 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14884c98-c60f-4223-89ba-d20ebc389fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:49.906487 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14884c98-c60f-4223-89ba-d20ebc389fb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:49.907254 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14884c98-c60f-4223-89ba-d20ebc389fb5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:49.908654 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14884c98-c60f-4223-89ba-d20ebc389fb5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:49.920089 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:49.925743 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c772ac8c-8d8f-4b46-ba95-e1fbce976266 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:49.959545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b6d39-e0f3-4de3-bca8-a3ecc67a81ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:49.960354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b6d39-e0f3-4de3-bca8-a3ecc67a81ca None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:49.960718 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b6d39-e0f3-4de3-bca8-a3ecc67a81ca None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:49.961002 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b6d39-e0f3-4de3-bca8-a3ecc67a81ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:49.963644 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b6d39-e0f3-4de3-bca8-a3ecc67a81ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:49.963810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b6d39-e0f3-4de3-bca8-a3ecc67a81ca None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:49.965444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 22/44] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:57:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:57:49.970815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b6d39-e0f3-4de3-bca8-a3ecc67a81ca None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:49.988855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 23/45] 10.222.0.16 () {60 vars in 1131 bytes} [Fri Jun 24 03:57:49 2022] POST /identity/v3/roles => generated 223 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:57:50.096480 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b6d39-e0f3-4de3-bca8-a3ecc67a81ca None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:50.135026 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29546986-6cdc-4850-849e-88c30c56d5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:50.135711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29546986-6cdc-4850-849e-88c30c56d5ac None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:50.136007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29546986-6cdc-4850-849e-88c30c56d5ac None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:50.136291 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29546986-6cdc-4850-849e-88c30c56d5ac None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:50.137033 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29546986-6cdc-4850-849e-88c30c56d5ac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:50.138545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29546986-6cdc-4850-849e-88c30c56d5ac None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:50.256045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 23/46] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jun 24 03:57:49 2022] POST /identity/v3/projects => generated 312 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 03:57:51.426750 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eebe50-ca40-49a2-95ef-9001c22994e8 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:51.427364 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eebe50-ca40-49a2-95ef-9001c22994e8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:51.427450 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eebe50-ca40-49a2-95ef-9001c22994e8 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:51.428851 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 24/47] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:57:51.434054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6dc7d-ac4e-4f52-bae4-88faf830d5d8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:51.434455 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6dc7d-ac4e-4f52-bae4-88faf830d5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:51.434654 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6dc7d-ac4e-4f52-bae4-88faf830d5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:51.451472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-36b6dc7d-ac4e-4f52-bae4-88faf830d5d8 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:57:51.452641 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-36b6dc7d-ac4e-4f52-bae4-88faf830d5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:57:51.463517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:51.517219 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 24/48] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:57:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:57:51.540881 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eebe50-ca40-49a2-95ef-9001c22994e8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:51.550055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:51.643507 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eebe50-ca40-49a2-95ef-9001c22994e8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:51.675908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df39dbf-9c96-4813-acf7-5e2c044a5ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:51.677279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df39dbf-9c96-4813-acf7-5e2c044a5ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:51.677510 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df39dbf-9c96-4813-acf7-5e2c044a5ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:51.677796 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df39dbf-9c96-4813-acf7-5e2c044a5ec0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:51.689454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df39dbf-9c96-4813-acf7-5e2c044a5ec0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:57:51.698856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-8df39dbf-9c96-4813-acf7-5e2c044a5ec0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:57:51.698856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:57:51.698856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:57:51.698856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:57:51.698856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:57:51.698856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:57:51.698856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:57:51.698856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:57:51.698856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:57:51.698856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:57:51.698856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:57:51.698856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:57:51.698856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:57:51.698856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:57:51.698856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:57:51.698856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:57:51.699826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:57:51.699826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:57:51.699826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:57:51.699826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:57:51.699826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:57:51.699826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:57:51.699826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:57:51.699826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:57:51.699826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:57:51.699826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:57:51.699826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:57:51.699826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:57:51.699826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:57:51.699826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:57:51.699826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:57:51.699826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:57:51.700599 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:57:51.700599 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:57:51.700599 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:57:51.700599 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:57:51.700599 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:57:51.700599 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:57:51.700599 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:57:51.700599 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:57:51.700599 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:57:51.700599 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:57:51.700599 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:57:51.700599 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:57:51.700599 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 25/49] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:57:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:57:51.706015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6dc7d-ac4e-4f52-bae4-88faf830d5d8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:51.804413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6dc7d-ac4e-4f52-bae4-88faf830d5d8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:51.838452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8879c-9a6a-4237-bd8f-a03e7fe6db4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:51.839130 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8879c-9a6a-4237-bd8f-a03e7fe6db4d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:51.839423 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8879c-9a6a-4237-bd8f-a03e7fe6db4d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:51.839711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece8879c-9a6a-4237-bd8f-a03e7fe6db4d None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:51.840468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8879c-9a6a-4237-bd8f-a03e7fe6db4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:57:51.840787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8879c-9a6a-4237-bd8f-a03e7fe6db4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:51.842190 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece8879c-9a6a-4237-bd8f-a03e7fe6db4d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:51.850998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 25/50] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:57:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:57:51.856607 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df39dbf-9c96-4813-acf7-5e2c044a5ec0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:51.922435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:51.957984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df39dbf-9c96-4813-acf7-5e2c044a5ec0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:51.992714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc0fe9-594e-4649-9da3-1532b48dc540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:51.993837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc0fe9-594e-4649-9da3-1532b48dc540 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:51.994124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc0fe9-594e-4649-9da3-1532b48dc540 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:51.994456 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc0fe9-594e-4649-9da3-1532b48dc540 None admin] PATH_INFO: `/v3/users/c531d2732de94f47a66f16db7047917f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:51.995670 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc0fe9-594e-4649-9da3-1532b48dc540 None admin] RBAC: Authorizing `identity:get_user(user_id=c531d2732de94f47a66f16db7047917f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:51.997145 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc0fe9-594e-4649-9da3-1532b48dc540 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:51.998887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 26/51] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:57:51 2022] GET /identity/v3/users/c531d2732de94f47a66f16db7047917f => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:57:52.004272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8879c-9a6a-4237-bd8f-a03e7fe6db4d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:52.016300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:52.107572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece8879c-9a6a-4237-bd8f-a03e7fe6db4d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:52.142708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d31fb3-493a-4d7c-a366-dcde1e5962b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:52.143503 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d31fb3-493a-4d7c-a366-dcde1e5962b3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:52.143790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d31fb3-493a-4d7c-a366-dcde1e5962b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:52.144070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d31fb3-493a-4d7c-a366-dcde1e5962b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:52.145483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d31fb3-493a-4d7c-a366-dcde1e5962b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:52.147601 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d31fb3-493a-4d7c-a366-dcde1e5962b3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:52.149241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 26/52] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:57:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:57:52.154932 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc0fe9-594e-4649-9da3-1532b48dc540 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:52.255804 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc0fe9-594e-4649-9da3-1532b48dc540 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:52.289653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2da67-70ab-40f4-b83d-a8044cdf870f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:52.290307 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2da67-70ab-40f4-b83d-a8044cdf870f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:52.290622 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2da67-70ab-40f4-b83d-a8044cdf870f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:52.290909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2da67-70ab-40f4-b83d-a8044cdf870f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:52.291739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c2da67-70ab-40f4-b83d-a8044cdf870f None admin] RBAC: Adding query filter params (role.id=f0f7e28cbe314921afdfb657b163436f, scope.domain.id=default, user.id=c531d2732de94f47a66f16db7047917f) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:57:52.292053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c2da67-70ab-40f4-b83d-a8044cdf870f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:52.293598 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c2da67-70ab-40f4-b83d-a8044cdf870f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:52.295253 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55a868-e4d9-41e5-ba6e-051649a9afaa None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:52.295551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55a868-e4d9-41e5-ba6e-051649a9afaa None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:52.295842 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55a868-e4d9-41e5-ba6e-051649a9afaa None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:52.295975 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:52.297223 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 27/53] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 03:57:52.302067 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4bcda-b1d4-4f8e-82a5-98bac53b632c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:52.302394 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4bcda-b1d4-4f8e-82a5-98bac53b632c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:52.302694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf4bcda-b1d4-4f8e-82a5-98bac53b632c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:52.305209 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 27/54] 10.222.0.16 () {56 vars in 1322 bytes} [Fri Jun 24 03:57:52 2022] GET /identity/v3/role_assignments?user.id=c531d2732de94f47a66f16db7047917f&scope.domain.id=default&role.id=f0f7e28cbe314921afdfb657b163436f => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:57:52.320183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ddf4bcda-b1d4-4f8e-82a5-98bac53b632c None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:57:52.321411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ddf4bcda-b1d4-4f8e-82a5-98bac53b632c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:57:52.400074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 28/55] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:57:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:57:52.423834 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2da67-70ab-40f4-b83d-a8044cdf870f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:52.528442 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c2da67-70ab-40f4-b83d-a8044cdf870f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:52.562572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b5572-f086-49d9-bc2b-24bff034623e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:52.563350 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b5572-f086-49d9-bc2b-24bff034623e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:52.563639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b5572-f086-49d9-bc2b-24bff034623e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:52.563916 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b5572-f086-49d9-bc2b-24bff034623e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:52.565093 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b5572-f086-49d9-bc2b-24bff034623e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:52.566577 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b5572-f086-49d9-bc2b-24bff034623e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:52.568224 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 28/56] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:57:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:57:52.573955 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf4bcda-b1d4-4f8e-82a5-98bac53b632c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:52.581907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:52.671994 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf4bcda-b1d4-4f8e-82a5-98bac53b632c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:52.704067 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076079b-61dd-4f9b-85bc-52e162476e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:52.704721 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076079b-61dd-4f9b-85bc-52e162476e80 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:52.705008 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076079b-61dd-4f9b-85bc-52e162476e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:52.705287 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076079b-61dd-4f9b-85bc-52e162476e80 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:52.705988 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076079b-61dd-4f9b-85bc-52e162476e80 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:52.707468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d076079b-61dd-4f9b-85bc-52e162476e80 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:52.797157 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 29/57] 10.222.0.16 () {60 vars in 1140 bytes} [Fri Jun 24 03:57:52 2022] POST /identity/v3/projects => generated 298 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:57:54.835846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7539f9-7192-4a3b-bf22-a8be1cc4d9c6 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:54.836469 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7539f9-7192-4a3b-bf22-a8be1cc4d9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:54.836469 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7539f9-7192-4a3b-bf22-a8be1cc4d9c6 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:54.837782 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 29/58] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:57:54.843009 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a6021-b18e-4e13-987f-a24322861931 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:54.843445 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a6021-b18e-4e13-987f-a24322861931 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:54.843580 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9a6021-b18e-4e13-987f-a24322861931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:54.859956 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9e9a6021-b18e-4e13-987f-a24322861931 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:57:54.861141 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9e9a6021-b18e-4e13-987f-a24322861931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:57:54.913738 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:54.927369 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 30/59] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:57:54.951749 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7539f9-7192-4a3b-bf22-a8be1cc4d9c6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:55.123661 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:55.157382 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7539f9-7192-4a3b-bf22-a8be1cc4d9c6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:55.190728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c2056-b980-49b4-a080-7d3ef8b1adfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:55.191515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c2056-b980-49b4-a080-7d3ef8b1adfd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:55.191802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c2056-b980-49b4-a080-7d3ef8b1adfd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:55.192195 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c2056-b980-49b4-a080-7d3ef8b1adfd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:55.193436 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c2056-b980-49b4-a080-7d3ef8b1adfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:55.194949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c2056-b980-49b4-a080-7d3ef8b1adfd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:55.196570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 30/60] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:57:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:57:55.202063 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a6021-b18e-4e13-987f-a24322861931 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:55.297863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9a6021-b18e-4e13-987f-a24322861931 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:55.329911 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf562a43-c471-40d4-8318-b8d3dc81c805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:55.330597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf562a43-c471-40d4-8318-b8d3dc81c805 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:55.330895 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf562a43-c471-40d4-8318-b8d3dc81c805 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:55.331173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf562a43-c471-40d4-8318-b8d3dc81c805 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:55.331890 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf562a43-c471-40d4-8318-b8d3dc81c805 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:55.333944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf562a43-c471-40d4-8318-b8d3dc81c805 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:55.416206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 31/61] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jun 24 03:57:55 2022] POST /identity/v3/users => generated 275 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:57:58.652878 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed6843-c7c9-45de-a6e1-2bfcca69d860 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:58.653414 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed6843-c7c9-45de-a6e1-2bfcca69d860 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:58.653506 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed6843-c7c9-45de-a6e1-2bfcca69d860 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:58.655022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 31/62] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:57:58.667038 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadd675-645f-417b-863e-ce88b8165090 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:58.667533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadd675-645f-417b-863e-ce88b8165090 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:58.667612 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadd675-645f-417b-863e-ce88b8165090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:58.690357 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7cadd675-645f-417b-863e-ce88b8165090 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:57:58.691598 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7cadd675-645f-417b-863e-ce88b8165090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:57:58.751404 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:58.759642 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ef925-edab-4885-be1c-113cb876fe7f None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:58.763935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ef925-edab-4885-be1c-113cb876fe7f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:58.763935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ef925-edab-4885-be1c-113cb876fe7f None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:58.763935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 32/63] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:57:58.766771 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef954b8-904f-4fda-8ee7-96cb06fda059 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:58.767066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef954b8-904f-4fda-8ee7-96cb06fda059 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:58.767352 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef954b8-904f-4fda-8ee7-96cb06fda059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:58.782606 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 32/64] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:57:58.795981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0ef954b8-904f-4fda-8ee7-96cb06fda059 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:57:58.798344 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0ef954b8-904f-4fda-8ee7-96cb06fda059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:57:58.809234 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadd675-645f-417b-863e-ce88b8165090 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:58.939826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 33/65] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:57:58.963601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea8eb5-0922-416f-aef0-a6f7034d7f2f None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:58.967679 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea8eb5-0922-416f-aef0-a6f7034d7f2f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:58.967679 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea8eb5-0922-416f-aef0-a6f7034d7f2f None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:58.967679 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 34/66] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 03:57:58.970795 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fa9a0-f9fa-44c6-85e9-3cd9b09855b2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:58.971130 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fa9a0-f9fa-44c6-85e9-3cd9b09855b2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:58.971406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5fa9a0-f9fa-44c6-85e9-3cd9b09855b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:58.978790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadd675-645f-417b-863e-ce88b8165090 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:58.994135 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-fe5fa9a0-f9fa-44c6-85e9-3cd9b09855b2 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:57:58.995595 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fe5fa9a0-f9fa-44c6-85e9-3cd9b09855b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:57:59.024316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fad884-b3e3-43a3-a8cd-8815b47ddd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:59.025387 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fad884-b3e3-43a3-a8cd-8815b47ddd82 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:59.025736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fad884-b3e3-43a3-a8cd-8815b47ddd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:59.026401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fad884-b3e3-43a3-a8cd-8815b47ddd82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:59.045502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fad884-b3e3-43a3-a8cd-8815b47ddd82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:57:59.053614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-72fad884-b3e3-43a3-a8cd-8815b47ddd82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:57:59.053614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:57:59.053614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:57:59.053614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:57:59.053614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:57:59.053614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:57:59.053614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:57:59.053614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:57:59.053614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:57:59.053614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:57:59.053614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:57:59.053614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:57:59.053614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:57:59.053614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:57:59.053614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:57:59.053614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:57:59.054615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:57:59.054615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:57:59.054615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:57:59.054615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:57:59.054615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:57:59.054615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:57:59.054615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:57:59.054615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:57:59.054615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:57:59.054615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:57:59.054615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:57:59.054615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:57:59.054615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:57:59.054615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:57:59.054615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:57:59.054615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:57:59.055401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:57:59.055401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:57:59.055401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:57:59.055401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:57:59.055401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:57:59.055401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:57:59.055401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:57:59.055401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:57:59.055401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:57:59.055401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:57:59.055401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:57:59.055401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:57:59.057763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 33/67] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:57:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:57:59.063286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fad884-b3e3-43a3-a8cd-8815b47ddd82 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:59.073335 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:59.088056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 35/68] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:57:59.089208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fa9a0-f9fa-44c6-85e9-3cd9b09855b2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:59.103681 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:59.195587 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fad884-b3e3-43a3-a8cd-8815b47ddd82 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:59.219283 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5fa9a0-f9fa-44c6-85e9-3cd9b09855b2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:59.231544 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48efc3c0-8012-4845-8d31-7438d6664d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:59.232353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efc3c0-8012-4845-8d31-7438d6664d07 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:59.232646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efc3c0-8012-4845-8d31-7438d6664d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:59.232932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48efc3c0-8012-4845-8d31-7438d6664d07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:59.243175 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48efc3c0-8012-4845-8d31-7438d6664d07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:57:59.251167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-48efc3c0-8012-4845-8d31-7438d6664d07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:57:59.251167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:57:59.251167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:57:59.251167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:57:59.251167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:57:59.251167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:57:59.251167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:57:59.251167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:57:59.251167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:57:59.251167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:57:59.251167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:57:59.251167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:57:59.251167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:57:59.251167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:57:59.251167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:57:59.251167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:57:59.252188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:57:59.252188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:57:59.252188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:57:59.252188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:57:59.252188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:57:59.252188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:57:59.252188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:57:59.252188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:57:59.252188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:57:59.252188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:57:59.252188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:57:59.252188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:57:59.252188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:57:59.252188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:57:59.252188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:57:59.252188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:57:59.253105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:57:59.253105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:57:59.253105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:57:59.253105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:57:59.253105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:57:59.253105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:57:59.253105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:57:59.253105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:57:59.253105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:57:59.253105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:57:59.253105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:57:59.253105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:57:59.253105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 34/69] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:57:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:57:59.253935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ef875-60d5-4387-8020-2b9c20a13813 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:59.253935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ef875-60d5-4387-8020-2b9c20a13813 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:59.254316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ef875-60d5-4387-8020-2b9c20a13813 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:59.255563 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 35/70] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:57:59.258526 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ef875-60d5-4387-8020-2b9c20a13813 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:59.259131 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dff08d-8afb-4780-8ffe-2eca48ab29b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:59.259817 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dff08d-8afb-4780-8ffe-2eca48ab29b1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:59.260110 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dff08d-8afb-4780-8ffe-2eca48ab29b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:59.260393 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dff08d-8afb-4780-8ffe-2eca48ab29b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:59.261154 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dff08d-8afb-4780-8ffe-2eca48ab29b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:57:59.261469 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dff08d-8afb-4780-8ffe-2eca48ab29b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:59.262913 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dff08d-8afb-4780-8ffe-2eca48ab29b1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:59.275534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 36/71] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:57:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:57:59.275534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8821ce1-8a02-4973-abe0-42530b74dc12 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:59.275534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8821ce1-8a02-4973-abe0-42530b74dc12 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:59.275534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8821ce1-8a02-4973-abe0-42530b74dc12 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:59.275534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 37/72] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:57:59.275534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8821ce1-8a02-4973-abe0-42530b74dc12 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:59.336298 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:59.354304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:59.361708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ef875-60d5-4387-8020-2b9c20a13813 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:59.381031 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8821ce1-8a02-4973-abe0-42530b74dc12 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:59.394972 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f6ba0-1d28-4a68-b750-9f88d2543f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:59.395834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f6ba0-1d28-4a68-b750-9f88d2543f1b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:59.396125 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f6ba0-1d28-4a68-b750-9f88d2543f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:59.396410 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f6ba0-1d28-4a68-b750-9f88d2543f1b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:59.406520 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f6ba0-1d28-4a68-b750-9f88d2543f1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:57:59.414725 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2111ffc-ed64-4fd0-b2e3-e837434ebb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:59.415392 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2111ffc-ed64-4fd0-b2e3-e837434ebb37 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:59.415683 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2111ffc-ed64-4fd0-b2e3-e837434ebb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:59.415968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2111ffc-ed64-4fd0-b2e3-e837434ebb37 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:59.416070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-dc2f6ba0-1d28-4a68-b750-9f88d2543f1b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:57:59.416070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:57:59.416070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:57:59.416070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:57:59.416070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:57:59.416070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:57:59.416070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:57:59.416070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:57:59.416070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:57:59.416070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:57:59.416070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:57:59.416070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:57:59.416070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:57:59.416070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:57:59.416070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:57:59.416070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:57:59.417030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:57:59.417030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:57:59.417030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:57:59.417030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:57:59.417030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:57:59.417030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:57:59.417030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:57:59.417030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:57:59.417030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:57:59.417030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:57:59.417030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:57:59.417030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:57:59.417030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:57:59.417030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:57:59.417030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:57:59.417030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:57:59.422735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:57:59.422735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:57:59.422735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:57:59.422735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:57:59.422735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:57:59.422735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:57:59.422735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:57:59.422735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:57:59.422735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:57:59.422735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:57:59.422735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:57:59.422735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:57:59.423360 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2111ffc-ed64-4fd0-b2e3-e837434ebb37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:57:59.423360 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2111ffc-ed64-4fd0-b2e3-e837434ebb37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:59.423483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 36/73] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:57:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:57:59.423550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2111ffc-ed64-4fd0-b2e3-e837434ebb37 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:59.423613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82362da9-4e06-4d28-a11f-3c20ac0f54ff None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:59.423613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82362da9-4e06-4d28-a11f-3c20ac0f54ff None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:59.423613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82362da9-4e06-4d28-a11f-3c20ac0f54ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:59.433323 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 38/74] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:57:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:57:59.435225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2111ffc-ed64-4fd0-b2e3-e837434ebb37 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:59.436066 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-82362da9-4e06-4d28-a11f-3c20ac0f54ff None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:57:59.437269 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-82362da9-4e06-4d28-a11f-3c20ac0f54ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:57:59.510466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 37/75] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:57:59.510466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b372c-34c2-4d6c-a410-a76e19c30f33 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:59.510466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b372c-34c2-4d6c-a410-a76e19c30f33 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:59.510466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b372c-34c2-4d6c-a410-a76e19c30f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:59.522419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c54b372c-34c2-4d6c-a410-a76e19c30f33 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:57:59.523608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c54b372c-34c2-4d6c-a410-a76e19c30f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:57:59.539353 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2111ffc-ed64-4fd0-b2e3-e837434ebb37 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:59.556914 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:59.572989 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ab604-d94e-4d2f-899c-147c2a9faaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:59.573862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ab604-d94e-4d2f-899c-147c2a9faaf3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:59.574155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ab604-d94e-4d2f-899c-147c2a9faaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:59.574470 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ab604-d94e-4d2f-899c-147c2a9faaf3 None admin] PATH_INFO: `/v3/users/1b3131ee9b304a72b169fab71c8af3d5` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:59.590012 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ab604-d94e-4d2f-899c-147c2a9faaf3 None admin] RBAC: Authorizing `identity:get_user(user_id=1b3131ee9b304a72b169fab71c8af3d5)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:59.590810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 38/76] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:57:59.591655 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b372c-34c2-4d6c-a410-a76e19c30f33 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:59.597748 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ab604-d94e-4d2f-899c-147c2a9faaf3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:59.600696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 39/77] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:57:59 2022] GET /identity/v3/users/1b3131ee9b304a72b169fab71c8af3d5 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:57:59.601398 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ab604-d94e-4d2f-899c-147c2a9faaf3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:59.608385 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:59.690768 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b372c-34c2-4d6c-a410-a76e19c30f33 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:59.702815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ab604-d94e-4d2f-899c-147c2a9faaf3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:59.726025 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53603b-b96b-42ef-9847-ce8779049049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:59.726730 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53603b-b96b-42ef-9847-ce8779049049 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:59.727024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53603b-b96b-42ef-9847-ce8779049049 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:59.727304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c53603b-b96b-42ef-9847-ce8779049049 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:59.728055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c53603b-b96b-42ef-9847-ce8779049049 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:57:59.728365 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c53603b-b96b-42ef-9847-ce8779049049 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:59.729767 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c53603b-b96b-42ef-9847-ce8779049049 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:59.737575 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 39/78] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:57:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:57:59.738378 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53603b-b96b-42ef-9847-ce8779049049 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:59.739671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca8681-8aa5-4f4b-87b3-f27bd575eeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:59.740718 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca8681-8aa5-4f4b-87b3-f27bd575eeea None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:59.741017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca8681-8aa5-4f4b-87b3-f27bd575eeea None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:59.741422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dca8681-8aa5-4f4b-87b3-f27bd575eeea None admin] PATH_INFO: `/v3/users/c531d2732de94f47a66f16db7047917f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:59.743714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca8681-8aa5-4f4b-87b3-f27bd575eeea None admin] RBAC: Authorizing `identity:get_user(user_id=c531d2732de94f47a66f16db7047917f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:59.745198 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dca8681-8aa5-4f4b-87b3-f27bd575eeea None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:59.750990 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 40/79] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:57:59 2022] GET /identity/v3/users/c531d2732de94f47a66f16db7047917f => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:57:59.750990 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca8681-8aa5-4f4b-87b3-f27bd575eeea None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:59.794114 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:59.814822 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:57:59.837515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c53603b-b96b-42ef-9847-ce8779049049 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:59.851085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dca8681-8aa5-4f4b-87b3-f27bd575eeea None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:57:59.873391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0da3ef-2658-41f4-a052-d4451df9e16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:59.874253 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0da3ef-2658-41f4-a052-d4451df9e16e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:59.874573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0da3ef-2658-41f4-a052-d4451df9e16e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:59.874861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0da3ef-2658-41f4-a052-d4451df9e16e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:59.886318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0da3ef-2658-41f4-a052-d4451df9e16e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:57:59.892160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d0fe4-1dcc-4af4-a28e-404121b7b5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:57:59.892814 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d0fe4-1dcc-4af4-a28e-404121b7b5b4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:57:59.893107 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d0fe4-1dcc-4af4-a28e-404121b7b5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:57:59.893393 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d0fe4-1dcc-4af4-a28e-404121b7b5b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:57:59.894354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-df0da3ef-2658-41f4-a052-d4451df9e16e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:57:59.894354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:57:59.894354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:57:59.894354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:57:59.894354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:57:59.894354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:57:59.894354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:57:59.894354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:57:59.894354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:57:59.894354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:57:59.894354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:57:59.894354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:57:59.894354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:57:59.894354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:57:59.894354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:57:59.894354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:57:59.895300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:57:59.895300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:57:59.895300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:57:59.895300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:57:59.895300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:57:59.895300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:57:59.895300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:57:59.895300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:57:59.895300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:57:59.895300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:57:59.895300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:57:59.895300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:57:59.895300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:57:59.895300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:57:59.895300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:57:59.895300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:57:59.896054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:57:59.896054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:57:59.896054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:57:59.896054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:57:59.896054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:57:59.896054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:57:59.896054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:57:59.896054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:57:59.896054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:57:59.896054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:57:59.896054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:57:59.896054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:57:59.896054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 40/80] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:57:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:57:59.896963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0da3ef-2658-41f4-a052-d4451df9e16e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:57:59.963891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d0fe4-1dcc-4af4-a28e-404121b7b5b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:57:59.965372 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d0fe4-1dcc-4af4-a28e-404121b7b5b4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:57:59.996802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0da3ef-2658-41f4-a052-d4451df9e16e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:00.009145 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:00.030620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c932b-ab08-4ad8-87ba-2f336e241212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:00.031420 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c932b-ab08-4ad8-87ba-2f336e241212 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:00.031705 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c932b-ab08-4ad8-87ba-2f336e241212 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:00.031984 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c932b-ab08-4ad8-87ba-2f336e241212 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:00.033186 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c932b-ab08-4ad8-87ba-2f336e241212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:00.034668 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c932b-ab08-4ad8-87ba-2f336e241212 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:00.036325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 41/81] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:57:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:00.040982 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c932b-ab08-4ad8-87ba-2f336e241212 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:00.042249 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:00.068975 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 41/82] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:57:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:00.069914 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d0fe4-1dcc-4af4-a28e-404121b7b5b4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:00.142262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c932b-ab08-4ad8-87ba-2f336e241212 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:00.168020 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d0fe4-1dcc-4af4-a28e-404121b7b5b4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:00.178394 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b887f-072c-4682-9389-8def207124f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:00.179867 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b887f-072c-4682-9389-8def207124f2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:00.180186 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b887f-072c-4682-9389-8def207124f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:00.180506 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b887f-072c-4682-9389-8def207124f2 None admin] PATH_INFO: `/v3/users/1b3131ee9b304a72b169fab71c8af3d5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:00.182000 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b887f-072c-4682-9389-8def207124f2 None admin] RBAC: Authorizing `identity:get_user(user_id=1b3131ee9b304a72b169fab71c8af3d5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:00.183837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b887f-072c-4682-9389-8def207124f2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:00.186019 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 42/83] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:00 2022] GET /identity/v3/users/1b3131ee9b304a72b169fab71c8af3d5 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:00.186835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b887f-072c-4682-9389-8def207124f2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:00.205433 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171f4ab-c307-4d6d-b4c4-32c2507e2e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:00.206088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171f4ab-c307-4d6d-b4c4-32c2507e2e08 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:00.206388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171f4ab-c307-4d6d-b4c4-32c2507e2e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:00.206682 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171f4ab-c307-4d6d-b4c4-32c2507e2e08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:00.221867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:00.259959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:00.274755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171f4ab-c307-4d6d-b4c4-32c2507e2e08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:00.276232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171f4ab-c307-4d6d-b4c4-32c2507e2e08 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:00.289325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b887f-072c-4682-9389-8def207124f2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:00.325652 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df089595-8013-4378-b8cb-ab95d14fe33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:00.326266 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df089595-8013-4378-b8cb-ab95d14fe33e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:00.326589 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df089595-8013-4378-b8cb-ab95d14fe33e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:00.326880 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df089595-8013-4378-b8cb-ab95d14fe33e None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:00.327636 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df089595-8013-4378-b8cb-ab95d14fe33e None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:00.327955 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df089595-8013-4378-b8cb-ab95d14fe33e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:00.329625 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df089595-8013-4378-b8cb-ab95d14fe33e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:00.337603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 43/84] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:00.338387 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df089595-8013-4378-b8cb-ab95d14fe33e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:00.374809 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 42/85] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:00.375495 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171f4ab-c307-4d6d-b4c4-32c2507e2e08 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:00.420733 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:00.443458 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df089595-8013-4378-b8cb-ab95d14fe33e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:00.474153 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171f4ab-c307-4d6d-b4c4-32c2507e2e08 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:00.476800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3171cfc-f499-4522-92b9-ca6a41370135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:00.477474 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3171cfc-f499-4522-92b9-ca6a41370135 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:00.477761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3171cfc-f499-4522-92b9-ca6a41370135 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:00.478039 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3171cfc-f499-4522-92b9-ca6a41370135 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:00.478793 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3171cfc-f499-4522-92b9-ca6a41370135 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:00.480260 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3171cfc-f499-4522-92b9-ca6a41370135 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:00.487021 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:00.510266 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e7197-74f7-4f9d-97fa-c45acc3be15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:00.511158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e7197-74f7-4f9d-97fa-c45acc3be15b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:00.511450 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e7197-74f7-4f9d-97fa-c45acc3be15b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:00.511739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e7197-74f7-4f9d-97fa-c45acc3be15b None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:00.512935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e7197-74f7-4f9d-97fa-c45acc3be15b None admin] RBAC: Authorizing `identity:get_project(project_id=40f51bfd8db04611b068f7edd7002004)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:00.514422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e7197-74f7-4f9d-97fa-c45acc3be15b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:00.516124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 43/86] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:00 2022] GET /identity/v3/projects/40f51bfd8db04611b068f7edd7002004 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:00.516896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e7197-74f7-4f9d-97fa-c45acc3be15b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:00.589839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 44/87] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jun 24 03:58:00 2022] POST /identity/v3/users => generated 289 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:58:00.589839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3171cfc-f499-4522-92b9-ca6a41370135 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:00.629090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e7197-74f7-4f9d-97fa-c45acc3be15b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:00.646170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:00.671738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219ced8-fd7a-4a25-ab2d-909e4110b2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:00.673525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219ced8-fd7a-4a25-ab2d-909e4110b2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:00.673525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219ced8-fd7a-4a25-ab2d-909e4110b2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:00.673853 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219ced8-fd7a-4a25-ab2d-909e4110b2a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:00.696578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3171cfc-f499-4522-92b9-ca6a41370135 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:00.741125 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b919527-2a00-4db6-940c-e21d2fc127db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:00.742150 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b919527-2a00-4db6-940c-e21d2fc127db None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:00.742333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b919527-2a00-4db6-940c-e21d2fc127db None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:00.742644 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b919527-2a00-4db6-940c-e21d2fc127db None admin] PATH_INFO: `/v3/users/1b3131ee9b304a72b169fab71c8af3d5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:00.744327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b919527-2a00-4db6-940c-e21d2fc127db None admin] RBAC: Authorizing `identity:get_user(user_id=1b3131ee9b304a72b169fab71c8af3d5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:00.746997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b919527-2a00-4db6-940c-e21d2fc127db None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:00.750317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 45/88] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:00 2022] GET /identity/v3/users/1b3131ee9b304a72b169fab71c8af3d5 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:00.750752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b919527-2a00-4db6-940c-e21d2fc127db None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:00.756438 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6219ced8-fd7a-4a25-ab2d-909e4110b2a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:00.757413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6219ced8-fd7a-4a25-ab2d-909e4110b2a7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:00.819379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:00.876505 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b919527-2a00-4db6-940c-e21d2fc127db None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:00.887152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 44/89] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:00.888964 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219ced8-fd7a-4a25-ab2d-909e4110b2a7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:00.906278 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:00.916762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf9abc-272b-4663-a1ec-61669622a55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:00.921338 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf9abc-272b-4663-a1ec-61669622a55a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:00.921338 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf9abc-272b-4663-a1ec-61669622a55a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:00.921338 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaf9abc-272b-4663-a1ec-61669622a55a None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:00.921338 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf9abc-272b-4663-a1ec-61669622a55a None admin] RBAC: Authorizing `identity:get_project(project_id=40f51bfd8db04611b068f7edd7002004)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:00.923298 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaf9abc-272b-4663-a1ec-61669622a55a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:00.925806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 46/90] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:00 2022] GET /identity/v3/projects/40f51bfd8db04611b068f7edd7002004 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:00.926877 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf9abc-272b-4663-a1ec-61669622a55a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:01.003011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219ced8-fd7a-4a25-ab2d-909e4110b2a7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:01.064518 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaf9abc-272b-4663-a1ec-61669622a55a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:01.066918 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2779e40-520a-4ecd-a14e-2a6f7caf1661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:01.068739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2779e40-520a-4ecd-a14e-2a6f7caf1661 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:01.068739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2779e40-520a-4ecd-a14e-2a6f7caf1661 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:01.068935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2779e40-520a-4ecd-a14e-2a6f7caf1661 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:01.069715 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2779e40-520a-4ecd-a14e-2a6f7caf1661 None admin] RBAC: Adding query filter params (role.id=d7aadc20cd4c4838ae5d7d8e8826abae, scope.project.id=40f51bfd8db04611b068f7edd7002004, user.id=1b3131ee9b304a72b169fab71c8af3d5) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:01.070030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2779e40-520a-4ecd-a14e-2a6f7caf1661 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:01.071600 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2779e40-520a-4ecd-a14e-2a6f7caf1661 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:01.077962 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 45/91] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:00 2022] GET /identity/v3/role_assignments?user.id=1b3131ee9b304a72b169fab71c8af3d5&scope.project.id=40f51bfd8db04611b068f7edd7002004&role.id=d7aadc20cd4c4838ae5d7d8e8826abae => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:01.087430 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2779e40-520a-4ecd-a14e-2a6f7caf1661 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:01.106766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:01.134775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78a007-dc75-4fe2-b1b3-99ec3cdfd41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:01.135961 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78a007-dc75-4fe2-b1b3-99ec3cdfd41a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:01.138190 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78a007-dc75-4fe2-b1b3-99ec3cdfd41a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:01.138758 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78a007-dc75-4fe2-b1b3-99ec3cdfd41a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:01.226855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:01.274221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2779e40-520a-4ecd-a14e-2a6f7caf1661 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:01.279216 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78a007-dc75-4fe2-b1b3-99ec3cdfd41a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:01.280723 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78a007-dc75-4fe2-b1b3-99ec3cdfd41a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:01.315376 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30382c65-ab3a-4f97-bf7b-573a8f88c4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:01.317152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30382c65-ab3a-4f97-bf7b-573a8f88c4a9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:01.318426 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30382c65-ab3a-4f97-bf7b-573a8f88c4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:01.318908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30382c65-ab3a-4f97-bf7b-573a8f88c4a9 None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:01.320496 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30382c65-ab3a-4f97-bf7b-573a8f88c4a9 None admin] RBAC: Authorizing `identity:get_project(project_id=40f51bfd8db04611b068f7edd7002004)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:01.322161 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30382c65-ab3a-4f97-bf7b-573a8f88c4a9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:01.325117 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 46/92] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:01 2022] GET /identity/v3/projects/40f51bfd8db04611b068f7edd7002004 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:01.326403 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30382c65-ab3a-4f97-bf7b-573a8f88c4a9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:01.446579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30382c65-ab3a-4f97-bf7b-573a8f88c4a9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:01.481506 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 47/93] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:01.486637 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78a007-dc75-4fe2-b1b3-99ec3cdfd41a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:01.529130 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a00e52-3f1c-4f41-98d2-7c42ea0d49e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:01.530186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a00e52-3f1c-4f41-98d2-7c42ea0d49e2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:01.530691 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a00e52-3f1c-4f41-98d2-7c42ea0d49e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:01.531169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a00e52-3f1c-4f41-98d2-7c42ea0d49e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:01.532201 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a00e52-3f1c-4f41-98d2-7c42ea0d49e2 None admin] RBAC: Adding query filter params (role.id=f0f7e28cbe314921afdfb657b163436f, scope.project.id=40f51bfd8db04611b068f7edd7002004, user.id=c531d2732de94f47a66f16db7047917f) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:01.533786 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a00e52-3f1c-4f41-98d2-7c42ea0d49e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:01.535499 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a00e52-3f1c-4f41-98d2-7c42ea0d49e2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:01.543728 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:01.548495 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 47/94] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:01 2022] GET /identity/v3/role_assignments?user.id=c531d2732de94f47a66f16db7047917f&scope.project.id=40f51bfd8db04611b068f7edd7002004&role.id=f0f7e28cbe314921afdfb657b163436f => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:01.549402 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a00e52-3f1c-4f41-98d2-7c42ea0d49e2 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:01.569223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:01.635732 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78a007-dc75-4fe2-b1b3-99ec3cdfd41a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:01.678075 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587b3d0-c555-41b8-bf19-3af039dcdff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:01.679345 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587b3d0-c555-41b8-bf19-3af039dcdff9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:01.679850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587b3d0-c555-41b8-bf19-3af039dcdff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:01.680676 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587b3d0-c555-41b8-bf19-3af039dcdff9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:01.681791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587b3d0-c555-41b8-bf19-3af039dcdff9 None admin] RBAC: Adding query filter params (role.id=a048041c27f44edf98a110d23138a327, scope.project.id=40f51bfd8db04611b068f7edd7002004, user.id=1b3131ee9b304a72b169fab71c8af3d5) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:01.682344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587b3d0-c555-41b8-bf19-3af039dcdff9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:01.684089 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587b3d0-c555-41b8-bf19-3af039dcdff9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:01.691177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 48/95] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:01 2022] GET /identity/v3/role_assignments?user.id=1b3131ee9b304a72b169fab71c8af3d5&scope.project.id=40f51bfd8db04611b068f7edd7002004&role.id=a048041c27f44edf98a110d23138a327 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:01.743580 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a00e52-3f1c-4f41-98d2-7c42ea0d49e2 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:01.794116 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d15ed1-c958-457c-a926-35fc3cc51167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:01.795822 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d15ed1-c958-457c-a926-35fc3cc51167 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:01.796310 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d15ed1-c958-457c-a926-35fc3cc51167 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:01.796782 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d15ed1-c958-457c-a926-35fc3cc51167 None admin] PATH_INFO: `/v3/projects/130b568f2ae44a118343de8697aec423` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:01.803513 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d15ed1-c958-457c-a926-35fc3cc51167 None admin] RBAC: Authorizing `identity:get_project(project_id=130b568f2ae44a118343de8697aec423)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:01.805621 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d15ed1-c958-457c-a926-35fc3cc51167 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:01.807889 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 48/96] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:01 2022] GET /identity/v3/projects/130b568f2ae44a118343de8697aec423 => generated 312 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:01.814245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587b3d0-c555-41b8-bf19-3af039dcdff9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:01.925037 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:01.939519 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587b3d0-c555-41b8-bf19-3af039dcdff9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:01.991761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19bb3d-4dbf-42fb-98c3-675e95c5c999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:01.992822 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19bb3d-4dbf-42fb-98c3-675e95c5c999 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:01.993571 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19bb3d-4dbf-42fb-98c3-675e95c5c999 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:01.994098 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19bb3d-4dbf-42fb-98c3-675e95c5c999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:01.995204 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f19bb3d-4dbf-42fb-98c3-675e95c5c999 None admin] RBAC: Adding query filter params (role.id=d7aadc20cd4c4838ae5d7d8e8826abae, scope.project.id=130b568f2ae44a118343de8697aec423, user.id=1b3131ee9b304a72b169fab71c8af3d5) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:01.995736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f19bb3d-4dbf-42fb-98c3-675e95c5c999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:01.997413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f19bb3d-4dbf-42fb-98c3-675e95c5c999 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:02.004041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 49/97] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:01 2022] GET /identity/v3/role_assignments?user.id=1b3131ee9b304a72b169fab71c8af3d5&scope.project.id=130b568f2ae44a118343de8697aec423&role.id=d7aadc20cd4c4838ae5d7d8e8826abae => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:03.205925 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a166181-2635-415e-aaab-dddbb34c4e8f None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:03.206966 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a166181-2635-415e-aaab-dddbb34c4e8f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:03.207471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a166181-2635-415e-aaab-dddbb34c4e8f None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:03.209720 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 49/98] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:03.215414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a3ed4-98df-4aba-bb43-1a666343fecb None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:03.216272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a3ed4-98df-4aba-bb43-1a666343fecb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:03.216771 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a3ed4-98df-4aba-bb43-1a666343fecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:03.238030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1e7a3ed4-98df-4aba-bb43-1a666343fecb None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:03.239759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1e7a3ed4-98df-4aba-bb43-1a666343fecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:03.330840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 50/99] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:03.380465 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a166181-2635-415e-aaab-dddbb34c4e8f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:03.478107 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:03.510681 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a166181-2635-415e-aaab-dddbb34c4e8f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:03.549480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ddd1fd-0afb-4f3a-a356-02543f443f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:03.550282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddd1fd-0afb-4f3a-a356-02543f443f7c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:03.550601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddd1fd-0afb-4f3a-a356-02543f443f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:03.550881 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ddd1fd-0afb-4f3a-a356-02543f443f7c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:03.561690 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ddd1fd-0afb-4f3a-a356-02543f443f7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:03.571524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-88ddd1fd-0afb-4f3a-a356-02543f443f7c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 03:58:03.571524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:03.571524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:03.571524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:03.571524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:03.571524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:03.571524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:03.571524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:03.571524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:03.571524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:03.571524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:03.571524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:03.571524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:03.571524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:03.571524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:03.571524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:03.572371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:03.572371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:03.572371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:03.572371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:03.572371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:03.572371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:03.572371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:03.572371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:03.572371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:03.572371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:03.572371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:03.572371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:03.572371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:03.572371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:03.572371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:03.572371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:03.573279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:03.573279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:03.573279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:03.573279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:03.573279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:03.573279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:03.573279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:03.573279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:03.573279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:03.573279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:03.573279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 03:58:03.573279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:03.573279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 50/100] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:03.580088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a3ed4-98df-4aba-bb43-1a666343fecb None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:03.625598 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddfd5d-d70d-4b38-b6ea-00673ab7b3d6 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:03.625912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddfd5d-d70d-4b38-b6ea-00673ab7b3d6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:03.626186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddfd5d-d70d-4b38-b6ea-00673ab7b3d6 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:03.627670 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 51/101] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:03.633424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b05657-db59-4208-a184-7b597703e1f3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:03.633725 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b05657-db59-4208-a184-7b597703e1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:03.634013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b05657-db59-4208-a184-7b597703e1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:03.662680 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a6b05657-db59-4208-a184-7b597703e1f3 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:03.663894 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a6b05657-db59-4208-a184-7b597703e1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:03.710451 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:03.733039 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a3ed4-98df-4aba-bb43-1a666343fecb None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:03.791169 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3488cf7-5935-45ec-9245-3f2860aead2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:03.792308 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3488cf7-5935-45ec-9245-3f2860aead2c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:03.792912 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3488cf7-5935-45ec-9245-3f2860aead2c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:03.793385 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3488cf7-5935-45ec-9245-3f2860aead2c None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:03.794364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3488cf7-5935-45ec-9245-3f2860aead2c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:03.794898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3488cf7-5935-45ec-9245-3f2860aead2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:03.796657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3488cf7-5935-45ec-9245-3f2860aead2c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:03.808681 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 52/102] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:03.811953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 51/103] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:03.819419 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b05657-db59-4208-a184-7b597703e1f3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:03.843661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3488cf7-5935-45ec-9245-3f2860aead2c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:03.969671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:04.039662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b05657-db59-4208-a184-7b597703e1f3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:04.054402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3488cf7-5935-45ec-9245-3f2860aead2c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:04.113369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62777fc6-8f79-44e4-8963-3ca7171b203c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:04.114289 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62777fc6-8f79-44e4-8963-3ca7171b203c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:04.114602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62777fc6-8f79-44e4-8963-3ca7171b203c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:04.114884 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62777fc6-8f79-44e4-8963-3ca7171b203c None admin] PATH_INFO: `/v3/users/e22bb34e90cd4ea498ea37049b75163d` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:04.117004 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdcdf91-f41a-42f8-bcbf-ae9a198fb763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:04.117801 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcdf91-f41a-42f8-bcbf-ae9a198fb763 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:04.118088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcdf91-f41a-42f8-bcbf-ae9a198fb763 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:04.118650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcdf91-f41a-42f8-bcbf-ae9a198fb763 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:04.121017 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdcdf91-f41a-42f8-bcbf-ae9a198fb763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:04.122518 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdcdf91-f41a-42f8-bcbf-ae9a198fb763 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:04.124153 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 52/104] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:04.131586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e473db6-48af-4314-8580-0bce0b0802f9 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:04.131881 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e473db6-48af-4314-8580-0bce0b0802f9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:04.132162 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e473db6-48af-4314-8580-0bce0b0802f9 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:04.133511 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 53/105] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:04.134538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e473db6-48af-4314-8580-0bce0b0802f9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:04.149949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62777fc6-8f79-44e4-8963-3ca7171b203c None admin] RBAC: Authorizing `identity:get_user(user_id=e22bb34e90cd4ea498ea37049b75163d)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:04.151485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62777fc6-8f79-44e4-8963-3ca7171b203c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:04.153237 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 53/106] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:03 2022] GET /identity/v3/users/e22bb34e90cd4ea498ea37049b75163d => generated 289 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:04.158793 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bce7f-7a1b-4b78-b963-123678df428a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:04.159089 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bce7f-7a1b-4b78-b963-123678df428a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:04.159380 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bce7f-7a1b-4b78-b963-123678df428a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:04.174190 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:04.197353 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-677bce7f-7a1b-4b78-b963-123678df428a None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:04.198646 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-677bce7f-7a1b-4b78-b963-123678df428a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:04.291962 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e473db6-48af-4314-8580-0bce0b0802f9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:04.320563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 54/107] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:04.322307 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bce7f-7a1b-4b78-b963-123678df428a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:04.330228 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bc040-2148-4f5c-8295-f8940afd51f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:04.330922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bc040-2148-4f5c-8295-f8940afd51f6 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:04.331214 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bc040-2148-4f5c-8295-f8940afd51f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:04.331493 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96bc040-2148-4f5c-8295-f8940afd51f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:04.332217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96bc040-2148-4f5c-8295-f8940afd51f6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:04.333842 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96bc040-2148-4f5c-8295-f8940afd51f6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:04.398738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:04.441279 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 54/108] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jun 24 03:58:04 2022] POST /identity/v3/projects => generated 302 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:58:04.443311 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bc040-2148-4f5c-8295-f8940afd51f6 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:04.451027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bce7f-7a1b-4b78-b963-123678df428a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:04.492917 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d34be5-3d6d-4f47-aaa2-de7dd43ec508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:04.493943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d34be5-3d6d-4f47-aaa2-de7dd43ec508 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:04.494237 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d34be5-3d6d-4f47-aaa2-de7dd43ec508 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:04.494595 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d34be5-3d6d-4f47-aaa2-de7dd43ec508 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:04.540044 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:04.572377 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d34be5-3d6d-4f47-aaa2-de7dd43ec508 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:04.573863 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d34be5-3d6d-4f47-aaa2-de7dd43ec508 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:04.583669 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96bc040-2148-4f5c-8295-f8940afd51f6 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:04.625157 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6aad28-3d29-4214-a830-d12d3e29868f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:04.626487 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6aad28-3d29-4214-a830-d12d3e29868f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:04.626979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6aad28-3d29-4214-a830-d12d3e29868f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:04.627458 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6aad28-3d29-4214-a830-d12d3e29868f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:04.640245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6aad28-3d29-4214-a830-d12d3e29868f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:04.649045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-db6aad28-3d29-4214-a830-d12d3e29868f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:04.649045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:04.649045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:04.649045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:04.649045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:04.649045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:04.649045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:04.649045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:04.649045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:04.649045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:04.649045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:04.649045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:04.649045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:04.649045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:04.649045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:04.649045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:04.649958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:04.649958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:04.649958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:04.649958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:04.649958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:04.649958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:04.649958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:04.649958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:04.649958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:04.649958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:04.649958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:04.649958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:04.649958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:04.649958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:04.649958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:04.649958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:04.650800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:04.650800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:04.650800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:04.650800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:04.650800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:04.650800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:04.650800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:04.650800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:04.650800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:04.650800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:04.650800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:04.650800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:04.653442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 55/109] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:04.655004 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54699853-6f30-4400-a1cd-7ce3b1500dcc None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:04.655502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54699853-6f30-4400-a1cd-7ce3b1500dcc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:04.655983 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54699853-6f30-4400-a1cd-7ce3b1500dcc None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:04.657481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 56/110] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:04.663665 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06ab02-6a69-44a2-91b6-5ceb70b0575c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:04.664221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:04.665043 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06ab02-6a69-44a2-91b6-5ceb70b0575c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:04.667414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06ab02-6a69-44a2-91b6-5ceb70b0575c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:04.706411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9c06ab02-6a69-44a2-91b6-5ceb70b0575c None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:04.707626 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9c06ab02-6a69-44a2-91b6-5ceb70b0575c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:04.745266 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 55/111] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:04.752363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d34be5-3d6d-4f47-aaa2-de7dd43ec508 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:04.842736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 57/112] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:04.843525 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06ab02-6a69-44a2-91b6-5ceb70b0575c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:04.915282 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:04.925061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d34be5-3d6d-4f47-aaa2-de7dd43ec508 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:04.963137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06ab02-6a69-44a2-91b6-5ceb70b0575c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:04.963991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78446ff0-92cd-4faf-a5dc-fdee4f6e8e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:04.964684 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78446ff0-92cd-4faf-a5dc-fdee4f6e8e41 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:04.965135 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78446ff0-92cd-4faf-a5dc-fdee4f6e8e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:04.965428 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78446ff0-92cd-4faf-a5dc-fdee4f6e8e41 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:04.967078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78446ff0-92cd-4faf-a5dc-fdee4f6e8e41 None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:04.967401 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78446ff0-92cd-4faf-a5dc-fdee4f6e8e41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:04.972329 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78446ff0-92cd-4faf-a5dc-fdee4f6e8e41 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:04.984285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 56/113] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:04.985740 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06865b-e004-497d-b78b-4c422a64d159 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:04.986637 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06865b-e004-497d-b78b-4c422a64d159 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:04.987043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06865b-e004-497d-b78b-4c422a64d159 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:04.988743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 57/114] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:04.990342 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06865b-e004-497d-b78b-4c422a64d159 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:05.007579 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626399b0-747f-49df-9f05-faecdbfda0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:05.008473 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626399b0-747f-49df-9f05-faecdbfda0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:05.008888 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626399b0-747f-49df-9f05-faecdbfda0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:05.009230 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626399b0-747f-49df-9f05-faecdbfda0c1 None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:05.010625 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626399b0-747f-49df-9f05-faecdbfda0c1 None admin] RBAC: Authorizing `identity:get_project(project_id=40f51bfd8db04611b068f7edd7002004)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:05.012186 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626399b0-747f-49df-9f05-faecdbfda0c1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:05.012532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:05.015067 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 58/115] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:04 2022] GET /identity/v3/projects/40f51bfd8db04611b068f7edd7002004 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:05.015925 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626399b0-747f-49df-9f05-faecdbfda0c1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:05.173178 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626399b0-747f-49df-9f05-faecdbfda0c1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:05.175283 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06865b-e004-497d-b78b-4c422a64d159 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:05.203692 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:05.218574 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181a404-ca87-46a1-ab71-03af023233b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:05.219755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181a404-ca87-46a1-ab71-03af023233b2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:05.220252 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181a404-ca87-46a1-ab71-03af023233b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:05.220716 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181a404-ca87-46a1-ab71-03af023233b2 None admin] PATH_INFO: `/v3/users/1b3131ee9b304a72b169fab71c8af3d5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:05.221555 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec74298-000c-49b9-8445-2efc3997b19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:05.222029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181a404-ca87-46a1-ab71-03af023233b2 None admin] RBAC: Authorizing `identity:get_user(user_id=1b3131ee9b304a72b169fab71c8af3d5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:05.222402 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec74298-000c-49b9-8445-2efc3997b19d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:05.222687 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec74298-000c-49b9-8445-2efc3997b19d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:05.222968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec74298-000c-49b9-8445-2efc3997b19d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:05.224223 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181a404-ca87-46a1-ab71-03af023233b2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:05.226405 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 59/116] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:05 2022] GET /identity/v3/users/1b3131ee9b304a72b169fab71c8af3d5 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:05.227658 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3841b4-2e7a-4231-ad18-f6e93489a587 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:05.228420 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3841b4-2e7a-4231-ad18-f6e93489a587 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:05.228710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3841b4-2e7a-4231-ad18-f6e93489a587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:05.237971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec74298-000c-49b9-8445-2efc3997b19d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:05.246099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-5ec74298-000c-49b9-8445-2efc3997b19d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:58:05.246099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:05.246099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:05.246099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:05.246099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:05.246099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:05.246099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:05.246099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:05.246099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:05.246099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:05.246099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:05.246099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:05.246099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:05.246099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:05.246099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:05.246099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:05.247045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:05.247045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:05.247045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:05.247045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:05.247045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:05.247045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:05.247045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:05.247045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:05.247045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:05.247045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:05.247045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:05.247045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:05.247045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:05.247045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:05.247045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:05.247045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:05.247815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:05.247815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:05.247815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:05.247815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:05.247815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:05.247815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:05.247815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:05.247815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:05.247815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:05.247815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:05.247815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:58:05.247815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:05.249715 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-fc3841b4-2e7a-4231-ad18-f6e93489a587 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:05.250928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fc3841b4-2e7a-4231-ad18-f6e93489a587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:05.252351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 58/117] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:58:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:05.252983 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec74298-000c-49b9-8445-2efc3997b19d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:05.327238 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 60/118] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:05.331737 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335c26c-4387-46a0-8ff8-53c00621dd1d None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:05.332031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335c26c-4387-46a0-8ff8-53c00621dd1d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:05.332312 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335c26c-4387-46a0-8ff8-53c00621dd1d None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:05.334831 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 61/119] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:05.339529 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335c26c-4387-46a0-8ff8-53c00621dd1d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:05.346838 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:05.372847 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec74298-000c-49b9-8445-2efc3997b19d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:05.409520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf48863-e89b-4b82-8760-692cd5ad6b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:05.410174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf48863-e89b-4b82-8760-692cd5ad6b52 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:05.410491 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf48863-e89b-4b82-8760-692cd5ad6b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:05.410775 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf48863-e89b-4b82-8760-692cd5ad6b52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:05.411724 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf48863-e89b-4b82-8760-692cd5ad6b52 None admin] RBAC: Adding query filter params (role.id=368ae945e96746faa1dc2ecbcefcf0f1, scope.project.id=40f51bfd8db04611b068f7edd7002004, user.id=e22bb34e90cd4ea498ea37049b75163d) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:05.412044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf48863-e89b-4b82-8760-692cd5ad6b52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:05.413518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf48863-e89b-4b82-8760-692cd5ad6b52 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:05.419199 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 59/120] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:05 2022] GET /identity/v3/role_assignments?user.id=e22bb34e90cd4ea498ea37049b75163d&scope.project.id=40f51bfd8db04611b068f7edd7002004&role.id=368ae945e96746faa1dc2ecbcefcf0f1 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:05.419982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf48863-e89b-4b82-8760-692cd5ad6b52 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:05.496078 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335c26c-4387-46a0-8ff8-53c00621dd1d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:05.530934 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:05.545466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d6e14-0716-407e-9222-d81be7f0a0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:05.546520 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d6e14-0716-407e-9222-d81be7f0a0af None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:05.547021 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d6e14-0716-407e-9222-d81be7f0a0af None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:05.547472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d6e14-0716-407e-9222-d81be7f0a0af None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:05.553568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d6e14-0716-407e-9222-d81be7f0a0af None admin] RBAC: Authorizing `identity:get_project(project_id=40f51bfd8db04611b068f7edd7002004)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:05.555309 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d6e14-0716-407e-9222-d81be7f0a0af None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:05.557856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 62/121] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:05 2022] GET /identity/v3/projects/40f51bfd8db04611b068f7edd7002004 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:05.561376 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02902ebc-1825-4f7b-b4cc-ea00893b20c4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:05.561376 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02902ebc-1825-4f7b-b4cc-ea00893b20c4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:05.561376 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02902ebc-1825-4f7b-b4cc-ea00893b20c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:05.561598 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf48863-e89b-4b82-8760-692cd5ad6b52 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:05.583849 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-02902ebc-1825-4f7b-b4cc-ea00893b20c4 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:05.585607 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-02902ebc-1825-4f7b-b4cc-ea00893b20c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:05.604157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d13e6b-4fbd-4ddc-b4b1-29982874fe3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:05.605230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d13e6b-4fbd-4ddc-b4b1-29982874fe3a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:05.605671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d13e6b-4fbd-4ddc-b4b1-29982874fe3a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:05.606467 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d13e6b-4fbd-4ddc-b4b1-29982874fe3a None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:05.607374 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d13e6b-4fbd-4ddc-b4b1-29982874fe3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:05.607835 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d13e6b-4fbd-4ddc-b4b1-29982874fe3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:05.609840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d13e6b-4fbd-4ddc-b4b1-29982874fe3a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:05.622686 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 60/122] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:05.623363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d13e6b-4fbd-4ddc-b4b1-29982874fe3a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:05.636794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:05.669431 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 63/123] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:05.670429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02902ebc-1825-4f7b-b4cc-ea00893b20c4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:05.744349 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d13e6b-4fbd-4ddc-b4b1-29982874fe3a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:05.791176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02902ebc-1825-4f7b-b4cc-ea00893b20c4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:05.792346 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c5b60-1a28-4298-bb7b-46a0a0b44ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:05.794538 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c5b60-1a28-4298-bb7b-46a0a0b44ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:05.795023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c5b60-1a28-4298-bb7b-46a0a0b44ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:05.795486 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c5b60-1a28-4298-bb7b-46a0a0b44ad7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:05.810355 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c5b60-1a28-4298-bb7b-46a0a0b44ad7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:05.813259 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:05.823200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-7b5c5b60-1a28-4298-bb7b-46a0a0b44ad7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:05.823200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:05.823200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:05.823200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:05.823200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:05.823200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:05.823200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:05.823200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:05.823200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:05.823200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:05.823200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:05.823200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:05.823200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:05.823200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:05.823200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:05.823200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:05.829425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:05.829425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:05.829425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:05.829425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:05.829425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:05.829425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:05.829425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:05.829425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:05.829425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:05.829425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:05.829425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:05.829425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:05.829425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:05.829425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:05.829425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:05.829425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:05.830316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:05.830316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:05.830316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:05.830316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:05.830316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:05.830316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:05.830316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:05.830316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:05.830316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:05.830316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:05.830316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:05.830316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:05.830316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 61/124] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:58:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:05.830316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c5b60-1a28-4298-bb7b-46a0a0b44ad7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:05.831188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5207cbc-0dff-4372-a350-6f6f0b7fc131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:05.831188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5207cbc-0dff-4372-a350-6f6f0b7fc131 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:05.831565 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5207cbc-0dff-4372-a350-6f6f0b7fc131 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:05.832015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5207cbc-0dff-4372-a350-6f6f0b7fc131 None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004/users/1b3131ee9b304a72b169fab71c8af3d5/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:05.834800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5207cbc-0dff-4372-a350-6f6f0b7fc131 None admin] RBAC: Authorizing `identity:create_grant(project_id=40f51bfd8db04611b068f7edd7002004, user_id=1b3131ee9b304a72b169fab71c8af3d5, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:05.836486 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5207cbc-0dff-4372-a350-6f6f0b7fc131 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:05.901803 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:05.914464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 64/125] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 03:58:05 2022] PUT /identity/v3/projects/40f51bfd8db04611b068f7edd7002004/users/1b3131ee9b304a72b169fab71c8af3d5/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:58:05.915680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5207cbc-0dff-4372-a350-6f6f0b7fc131 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:05.963627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c5b60-1a28-4298-bb7b-46a0a0b44ad7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:06.052534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29276707-e6a5-45ec-a9bc-80ce0447dee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:06.053436 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29276707-e6a5-45ec-a9bc-80ce0447dee9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:06.053726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29276707-e6a5-45ec-a9bc-80ce0447dee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:06.054008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29276707-e6a5-45ec-a9bc-80ce0447dee9 None admin] PATH_INFO: `/v3/users/c531d2732de94f47a66f16db7047917f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:06.059580 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29276707-e6a5-45ec-a9bc-80ce0447dee9 None admin] RBAC: Authorizing `identity:get_user(user_id=c531d2732de94f47a66f16db7047917f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:06.061110 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29276707-e6a5-45ec-a9bc-80ce0447dee9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:06.063116 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 62/126] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:05 2022] GET /identity/v3/users/c531d2732de94f47a66f16db7047917f => generated 247 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:06.063739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29276707-e6a5-45ec-a9bc-80ce0447dee9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:06.137758 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5207cbc-0dff-4372-a350-6f6f0b7fc131 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:06.184254 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78696c4-b14c-476b-84ff-c2e60119012d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:06.185063 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78696c4-b14c-476b-84ff-c2e60119012d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:06.185357 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78696c4-b14c-476b-84ff-c2e60119012d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:06.185636 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78696c4-b14c-476b-84ff-c2e60119012d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:06.203440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78696c4-b14c-476b-84ff-c2e60119012d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:06.218261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-c78696c4-b14c-476b-84ff-c2e60119012d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:06.218261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:06.218261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:06.218261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:06.218261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:06.218261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:06.218261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:06.218261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:06.218261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:06.218261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:06.218261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:06.218261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:06.218261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:06.218261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:06.218261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:06.218261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:06.219206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:06.219206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:06.219206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:06.219206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:06.219206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:06.219206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:06.219206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:06.219206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:06.219206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:06.219206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:06.219206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:06.219206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:06.219206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:06.219206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:06.219206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:06.219206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:06.219968 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:06.219968 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:06.219968 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:06.219968 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:06.219968 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:06.219968 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:06.219968 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:06.219968 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:06.219968 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:06.219968 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:06.219968 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:06.219968 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:06.221638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 65/127] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:06.224453 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78696c4-b14c-476b-84ff-c2e60119012d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:06.233779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29276707-e6a5-45ec-a9bc-80ce0447dee9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:06.284305 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:06.297034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:06.309594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42002555-a7ef-43c6-8218-49151ca42322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:06.310402 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42002555-a7ef-43c6-8218-49151ca42322 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:06.310706 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42002555-a7ef-43c6-8218-49151ca42322 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:06.310990 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42002555-a7ef-43c6-8218-49151ca42322 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:06.311751 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42002555-a7ef-43c6-8218-49151ca42322 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:06.312070 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42002555-a7ef-43c6-8218-49151ca42322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:06.313487 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42002555-a7ef-43c6-8218-49151ca42322 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:06.325017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 63/128] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:58:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:06.325735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42002555-a7ef-43c6-8218-49151ca42322 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:06.367151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78696c4-b14c-476b-84ff-c2e60119012d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:06.417000 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e2ee9-1026-4fd6-9077-8124a6a10fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:06.417891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e2ee9-1026-4fd6-9077-8124a6a10fce None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:06.418186 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e2ee9-1026-4fd6-9077-8124a6a10fce None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:06.418498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e2ee9-1026-4fd6-9077-8124a6a10fce None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:06.419741 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e2ee9-1026-4fd6-9077-8124a6a10fce None admin] RBAC: Authorizing `identity:get_project(project_id=40f51bfd8db04611b068f7edd7002004)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:06.421217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e2ee9-1026-4fd6-9077-8124a6a10fce None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:06.423141 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 66/129] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:06 2022] GET /identity/v3/projects/40f51bfd8db04611b068f7edd7002004 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:06.423743 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e2ee9-1026-4fd6-9077-8124a6a10fce None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:06.468187 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42002555-a7ef-43c6-8218-49151ca42322 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:06.508352 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44272b-b7c7-4745-8737-618f1fd1f6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:06.509945 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44272b-b7c7-4745-8737-618f1fd1f6fc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:06.510239 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44272b-b7c7-4745-8737-618f1fd1f6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:06.510762 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44272b-b7c7-4745-8737-618f1fd1f6fc None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:06.513062 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44272b-b7c7-4745-8737-618f1fd1f6fc None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:06.513394 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44272b-b7c7-4745-8737-618f1fd1f6fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:06.515752 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44272b-b7c7-4745-8737-618f1fd1f6fc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:06.535557 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 64/130] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:06.536289 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44272b-b7c7-4745-8737-618f1fd1f6fc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:06.626100 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e2ee9-1026-4fd6-9077-8124a6a10fce None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:06.644116 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:06.651583 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:06.700049 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bae80a-04bb-4cd6-a498-083cc1ceb99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:06.700391 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44272b-b7c7-4745-8737-618f1fd1f6fc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:06.700947 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bae80a-04bb-4cd6-a498-083cc1ceb99d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:06.701247 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bae80a-04bb-4cd6-a498-083cc1ceb99d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:06.701535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bae80a-04bb-4cd6-a498-083cc1ceb99d None admin] PATH_INFO: `/v3/users/1b3131ee9b304a72b169fab71c8af3d5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:06.707147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bae80a-04bb-4cd6-a498-083cc1ceb99d None admin] RBAC: Authorizing `identity:get_user(user_id=1b3131ee9b304a72b169fab71c8af3d5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:06.708828 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bae80a-04bb-4cd6-a498-083cc1ceb99d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:06.710758 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 67/131] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:06 2022] GET /identity/v3/users/1b3131ee9b304a72b169fab71c8af3d5 => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:06.711384 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bae80a-04bb-4cd6-a498-083cc1ceb99d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:06.749298 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c5dec-6d3f-4a40-b4ef-003bea469232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:06.751486 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c5dec-6d3f-4a40-b4ef-003bea469232 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:06.751785 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c5dec-6d3f-4a40-b4ef-003bea469232 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:06.752067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c5dec-6d3f-4a40-b4ef-003bea469232 None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004/users/c531d2732de94f47a66f16db7047917f/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:06.756984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c5dec-6d3f-4a40-b4ef-003bea469232 None admin] RBAC: Authorizing `identity:create_grant(project_id=40f51bfd8db04611b068f7edd7002004, user_id=c531d2732de94f47a66f16db7047917f, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:06.760117 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c5dec-6d3f-4a40-b4ef-003bea469232 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:06.847289 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bae80a-04bb-4cd6-a498-083cc1ceb99d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:06.868239 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 65/132] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 03:58:06 2022] PUT /identity/v3/projects/40f51bfd8db04611b068f7edd7002004/users/c531d2732de94f47a66f16db7047917f/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:58:06.869256 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c5dec-6d3f-4a40-b4ef-003bea469232 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:06.934486 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf478aa-544d-4133-874c-5dfc978d2dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:06.936656 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf478aa-544d-4133-874c-5dfc978d2dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:06.936952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf478aa-544d-4133-874c-5dfc978d2dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:06.937240 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf478aa-544d-4133-874c-5dfc978d2dc8 None admin] PATH_INFO: `/v3/users/1b3131ee9b304a72b169fab71c8af3d5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:06.939932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf478aa-544d-4133-874c-5dfc978d2dc8 None admin] RBAC: Authorizing `identity:get_user(user_id=1b3131ee9b304a72b169fab71c8af3d5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:06.947994 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf478aa-544d-4133-874c-5dfc978d2dc8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:06.952934 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 68/133] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:06 2022] GET /identity/v3/users/1b3131ee9b304a72b169fab71c8af3d5 => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:06.958828 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf478aa-544d-4133-874c-5dfc978d2dc8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:07.063221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:07.124408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c5dec-6d3f-4a40-b4ef-003bea469232 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:07.131363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf478aa-544d-4133-874c-5dfc978d2dc8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:07.154795 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:07.183044 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ccb58a-41e3-4811-8c18-b6a65d0900de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:07.183935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccb58a-41e3-4811-8c18-b6a65d0900de None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:07.184224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccb58a-41e3-4811-8c18-b6a65d0900de None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:07.184557 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccb58a-41e3-4811-8c18-b6a65d0900de None admin] PATH_INFO: `/v3/projects/130b568f2ae44a118343de8697aec423` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:07.186505 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccb58a-41e3-4811-8c18-b6a65d0900de None admin] RBAC: Authorizing `identity:get_project(project_id=130b568f2ae44a118343de8697aec423)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:07.188002 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccb58a-41e3-4811-8c18-b6a65d0900de None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:07.191798 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 69/134] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:06 2022] GET /identity/v3/projects/130b568f2ae44a118343de8697aec423 => generated 312 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:07.194173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4271dc-843b-4e59-99b9-f845a2aca954 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:07.194980 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4271dc-843b-4e59-99b9-f845a2aca954 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:07.195483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4271dc-843b-4e59-99b9-f845a2aca954 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:07.198677 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 70/135] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:07.199395 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4271dc-843b-4e59-99b9-f845a2aca954 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:07.204479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af3c8f-a7c5-4fb7-aecf-e024451ccdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:07.205357 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af3c8f-a7c5-4fb7-aecf-e024451ccdce None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:07.205644 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af3c8f-a7c5-4fb7-aecf-e024451ccdce None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:07.205927 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af3c8f-a7c5-4fb7-aecf-e024451ccdce None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:07.209745 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af3c8f-a7c5-4fb7-aecf-e024451ccdce None admin] RBAC: Authorizing `identity:get_project(project_id=40f51bfd8db04611b068f7edd7002004)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:07.209745 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af3c8f-a7c5-4fb7-aecf-e024451ccdce None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:07.211013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 66/136] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:06 2022] GET /identity/v3/projects/40f51bfd8db04611b068f7edd7002004 => generated 298 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:07.212008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d978b-5d28-4242-afdb-c697c9153bf3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:07.212301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d978b-5d28-4242-afdb-c697c9153bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:07.212586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d978b-5d28-4242-afdb-c697c9153bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:07.234403 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-841d978b-5d28-4242-afdb-c697c9153bf3 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:07.236283 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-841d978b-5d28-4242-afdb-c697c9153bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:07.320163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 67/137] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:07.321515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d978b-5d28-4242-afdb-c697c9153bf3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:07.375058 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4271dc-843b-4e59-99b9-f845a2aca954 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:07.396396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:07.423314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e7e36-48cc-4d00-a401-5d1d4f347483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:07.424222 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e7e36-48cc-4d00-a401-5d1d4f347483 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:07.424560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e7e36-48cc-4d00-a401-5d1d4f347483 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:07.424788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e7e36-48cc-4d00-a401-5d1d4f347483 None admin] PATH_INFO: `/v3/projects/130b568f2ae44a118343de8697aec423/users/1b3131ee9b304a72b169fab71c8af3d5/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:07.426997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e7e36-48cc-4d00-a401-5d1d4f347483 None admin] RBAC: Authorizing `identity:create_grant(project_id=130b568f2ae44a118343de8697aec423, user_id=1b3131ee9b304a72b169fab71c8af3d5, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:07.428467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e7e36-48cc-4d00-a401-5d1d4f347483 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:07.446960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d978b-5d28-4242-afdb-c697c9153bf3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:07.456331 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:07.477760 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ada66-af1c-4c86-968c-91cc5b72bfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:07.478753 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ada66-af1c-4c86-968c-91cc5b72bfc6 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:07.479041 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ada66-af1c-4c86-968c-91cc5b72bfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:07.479324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ada66-af1c-4c86-968c-91cc5b72bfc6 None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004/users/1b3131ee9b304a72b169fab71c8af3d5/roles/a048041c27f44edf98a110d23138a327` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:07.481957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ada66-af1c-4c86-968c-91cc5b72bfc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=40f51bfd8db04611b068f7edd7002004, user_id=1b3131ee9b304a72b169fab71c8af3d5, role_id=a048041c27f44edf98a110d23138a327)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:07.483567 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ada66-af1c-4c86-968c-91cc5b72bfc6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:07.502959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 71/138] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 03:58:07 2022] PUT /identity/v3/projects/130b568f2ae44a118343de8697aec423/users/1b3131ee9b304a72b169fab71c8af3d5/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:58:07.503767 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e7e36-48cc-4d00-a401-5d1d4f347483 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:07.555961 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 68/139] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 03:58:07 2022] PUT /identity/v3/projects/40f51bfd8db04611b068f7edd7002004/users/1b3131ee9b304a72b169fab71c8af3d5/roles/a048041c27f44edf98a110d23138a327 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:58:07.707468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e7e36-48cc-4d00-a401-5d1d4f347483 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:07.761366 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec75b93-73b1-4d95-b5c7-a413569c30c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:07.762205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec75b93-73b1-4d95-b5c7-a413569c30c6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:07.762519 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec75b93-73b1-4d95-b5c7-a413569c30c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:07.762801 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec75b93-73b1-4d95-b5c7-a413569c30c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:07.764162 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec75b93-73b1-4d95-b5c7-a413569c30c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:07.765688 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec75b93-73b1-4d95-b5c7-a413569c30c6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:07.771741 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 72/140] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:07.777422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ada66-af1c-4c86-968c-91cc5b72bfc6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:07.878249 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026b658-1c21-44d8-80cf-454df2dfd1d2 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:07.878586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026b658-1c21-44d8-80cf-454df2dfd1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:07.878839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026b658-1c21-44d8-80cf-454df2dfd1d2 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:07.880801 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 73/141] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:07.886346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc985121-c464-4a87-a769-26942d3ace30 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:07.886668 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc985121-c464-4a87-a769-26942d3ace30 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:07.886950 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc985121-c464-4a87-a769-26942d3ace30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:07.911518 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-dc985121-c464-4a87-a769-26942d3ace30 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:07.911695 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ada66-af1c-4c86-968c-91cc5b72bfc6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:07.913303 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dc985121-c464-4a87-a769-26942d3ace30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:07.959707 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d25e1-3613-47c3-884f-909be4bb7ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:07.960322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d25e1-3613-47c3-884f-909be4bb7ba4 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:07.960612 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d25e1-3613-47c3-884f-909be4bb7ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:07.960895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d25e1-3613-47c3-884f-909be4bb7ba4 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:07.961651 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d25e1-3613-47c3-884f-909be4bb7ba4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:07.963178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d25e1-3613-47c3-884f-909be4bb7ba4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:07.974986 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:08.039128 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 74/142] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 24 03:58:08.066071 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc985121-c464-4a87-a769-26942d3ace30 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:08.139679 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 69/143] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jun 24 03:58:07 2022] POST /identity/v3/users => generated 283 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:58:08.207446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc985121-c464-4a87-a769-26942d3ace30 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:08.264201 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eef219-8dce-48df-b70e-6f6327dd19a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:08.264945 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eef219-8dce-48df-b70e-6f6327dd19a1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:08.265237 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eef219-8dce-48df-b70e-6f6327dd19a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:08.265516 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eef219-8dce-48df-b70e-6f6327dd19a1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:08.286346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eef219-8dce-48df-b70e-6f6327dd19a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:08.294674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-85eef219-8dce-48df-b70e-6f6327dd19a1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 03:58:08.294674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:08.294674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:08.294674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:08.294674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:08.294674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:08.294674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:08.294674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:08.294674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:08.294674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:08.294674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:08.294674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:08.294674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:08.294674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:08.294674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:08.294674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:08.295585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:08.295585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:08.295585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:08.295585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:08.295585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:08.295585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:08.295585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:08.295585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:08.295585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:08.295585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:08.295585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:08.295585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:08.295585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:08.295585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:08.295585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:08.295585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:08.296370 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:08.296370 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:08.296370 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:08.296370 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:08.296370 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:08.296370 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:08.296370 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:08.296370 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:08.296370 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:08.296370 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:08.296370 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 03:58:08.296370 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:08.296370 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 75/144] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:58:08.302724 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d25e1-3613-47c3-884f-909be4bb7ba4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:08.340419 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:08.421748 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d25e1-3613-47c3-884f-909be4bb7ba4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:08.465711 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dd4ec-1321-452e-8d64-a1fb4ea521f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:08.466757 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dd4ec-1321-452e-8d64-a1fb4ea521f0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:08.467254 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dd4ec-1321-452e-8d64-a1fb4ea521f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:08.467751 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dd4ec-1321-452e-8d64-a1fb4ea521f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:08.468737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188dd4ec-1321-452e-8d64-a1fb4ea521f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:08.469251 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188dd4ec-1321-452e-8d64-a1fb4ea521f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:08.470908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188dd4ec-1321-452e-8d64-a1fb4ea521f0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:08.484153 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 70/145] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:08.490634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eef219-8dce-48df-b70e-6f6327dd19a1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:08.543517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:08.636229 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eef219-8dce-48df-b70e-6f6327dd19a1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:08.679653 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20323ac-7bec-4919-8f23-e35ae4f72466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:08.680899 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20323ac-7bec-4919-8f23-e35ae4f72466 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:08.681386 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20323ac-7bec-4919-8f23-e35ae4f72466 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:08.681883 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20323ac-7bec-4919-8f23-e35ae4f72466 None admin] PATH_INFO: `/v3/users/e22bb34e90cd4ea498ea37049b75163d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:08.687094 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20323ac-7bec-4919-8f23-e35ae4f72466 None admin] RBAC: Authorizing `identity:get_user(user_id=e22bb34e90cd4ea498ea37049b75163d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:08.688795 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20323ac-7bec-4919-8f23-e35ae4f72466 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:08.690711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 76/146] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:08 2022] GET /identity/v3/users/e22bb34e90cd4ea498ea37049b75163d => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:08.696477 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dd4ec-1321-452e-8d64-a1fb4ea521f0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:08.719774 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f9de0-66ac-4f28-84b5-d97f730bf60d None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:08.720464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f9de0-66ac-4f28-84b5-d97f730bf60d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:08.720944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f9de0-66ac-4f28-84b5-d97f730bf60d None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:08.722564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 77/147] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 03:58:08.729065 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8bf88-d29f-4402-a47d-c6414723f1d7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:08.729597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8bf88-d29f-4402-a47d-c6414723f1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:08.730094 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa8bf88-d29f-4402-a47d-c6414723f1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:08.757648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9aa8bf88-d29f-4402-a47d-c6414723f1d7 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:08.759525 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9aa8bf88-d29f-4402-a47d-c6414723f1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:08.852239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 78/148] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:58:08.860441 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:08.878181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dd4ec-1321-452e-8d64-a1fb4ea521f0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:08.883607 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8bf88-d29f-4402-a47d-c6414723f1d7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:08.921355 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:08.943136 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905ead7-f680-4bd9-8268-04822c8ed389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:08.944029 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905ead7-f680-4bd9-8268-04822c8ed389 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:08.944313 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905ead7-f680-4bd9-8268-04822c8ed389 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:08.944590 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905ead7-f680-4bd9-8268-04822c8ed389 None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:08.948799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905ead7-f680-4bd9-8268-04822c8ed389 None admin] RBAC: Authorizing `identity:get_project(project_id=40f51bfd8db04611b068f7edd7002004)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:08.950278 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905ead7-f680-4bd9-8268-04822c8ed389 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:08.957190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 71/149] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:08 2022] GET /identity/v3/projects/40f51bfd8db04611b068f7edd7002004 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:08.960887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905ead7-f680-4bd9-8268-04822c8ed389 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:09.053100 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa8bf88-d29f-4402-a47d-c6414723f1d7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:09.123726 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbde4db-a463-4459-bc5d-1284fd2e1882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:09.124534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbde4db-a463-4459-bc5d-1284fd2e1882 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:09.124811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbde4db-a463-4459-bc5d-1284fd2e1882 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:09.125082 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbde4db-a463-4459-bc5d-1284fd2e1882 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:09.147769 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbde4db-a463-4459-bc5d-1284fd2e1882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:09.155594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-7cbde4db-a463-4459-bc5d-1284fd2e1882 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:09.155594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:09.155594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:09.155594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:09.155594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:09.155594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:09.155594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:09.155594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:09.155594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:09.155594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:09.155594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:09.155594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:09.155594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:09.155594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:09.155594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:09.155594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:09.156454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:09.156454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:09.156454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:09.156454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:09.156454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:09.156454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:09.156454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:09.156454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:09.156454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:09.156454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:09.156454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:09.156454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:09.156454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:09.156454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:09.156454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:09.156454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:09.157224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:09.157224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:09.157224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:09.157224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:09.157224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:09.157224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:09.157224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:09.157224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:09.157224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:09.157224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:09.157224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:09.157224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:09.158278 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 79/150] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:58:09.167345 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905ead7-f680-4bd9-8268-04822c8ed389 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:09.169330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbde4db-a463-4459-bc5d-1284fd2e1882 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:09.214146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e2975-139d-4b06-9e4d-6d8b9a374867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:09.215081 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e2975-139d-4b06-9e4d-6d8b9a374867 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:09.215367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e2975-139d-4b06-9e4d-6d8b9a374867 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:09.215650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e2975-139d-4b06-9e4d-6d8b9a374867 None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004/users/e22bb34e90cd4ea498ea37049b75163d/roles/368ae945e96746faa1dc2ecbcefcf0f1` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:09.218348 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e2975-139d-4b06-9e4d-6d8b9a374867 None admin] RBAC: Authorizing `identity:create_grant(project_id=40f51bfd8db04611b068f7edd7002004, user_id=e22bb34e90cd4ea498ea37049b75163d, role_id=368ae945e96746faa1dc2ecbcefcf0f1)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:09.220582 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e2975-139d-4b06-9e4d-6d8b9a374867 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:09.302152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 72/151] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 03:58:08 2022] PUT /identity/v3/projects/40f51bfd8db04611b068f7edd7002004/users/e22bb34e90cd4ea498ea37049b75163d/roles/368ae945e96746faa1dc2ecbcefcf0f1 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:58:09.311959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:09.326694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbde4db-a463-4459-bc5d-1284fd2e1882 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:09.368198 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1f57f-74ca-4ecf-876e-5eefc643c80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:09.369384 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1f57f-74ca-4ecf-876e-5eefc643c80d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:09.369811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1f57f-74ca-4ecf-876e-5eefc643c80d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:09.370293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f1f57f-74ca-4ecf-876e-5eefc643c80d None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:09.371215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1f57f-74ca-4ecf-876e-5eefc643c80d None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:09.371689 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1f57f-74ca-4ecf-876e-5eefc643c80d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:09.373274 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f1f57f-74ca-4ecf-876e-5eefc643c80d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:09.383290 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 80/152] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:09.389273 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e2975-139d-4b06-9e4d-6d8b9a374867 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:09.411369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:09.516619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e2975-139d-4b06-9e4d-6d8b9a374867 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:09.554386 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54470ecf-b275-4982-9e34-fa698f2342b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:09.555677 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54470ecf-b275-4982-9e34-fa698f2342b3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:09.556115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54470ecf-b275-4982-9e34-fa698f2342b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:09.556532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54470ecf-b275-4982-9e34-fa698f2342b3 None admin] PATH_INFO: `/v3/users/1b3131ee9b304a72b169fab71c8af3d5` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:09.557922 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54470ecf-b275-4982-9e34-fa698f2342b3 None admin] RBAC: Authorizing `identity:get_user(user_id=1b3131ee9b304a72b169fab71c8af3d5)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:09.559565 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54470ecf-b275-4982-9e34-fa698f2342b3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:09.561658 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 73/153] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:09 2022] GET /identity/v3/users/1b3131ee9b304a72b169fab71c8af3d5 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:09.567662 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1f57f-74ca-4ecf-876e-5eefc643c80d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:09.688645 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f1f57f-74ca-4ecf-876e-5eefc643c80d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:09.727997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274ac3c-5b5d-451c-9890-4d9fa137c681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:09.728976 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274ac3c-5b5d-451c-9890-4d9fa137c681 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:09.729399 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274ac3c-5b5d-451c-9890-4d9fa137c681 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:09.729858 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274ac3c-5b5d-451c-9890-4d9fa137c681 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:09.755373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:09.807310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274ac3c-5b5d-451c-9890-4d9fa137c681 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:09.809212 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274ac3c-5b5d-451c-9890-4d9fa137c681 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:09.921173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 81/154] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:09.927267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54470ecf-b275-4982-9e34-fa698f2342b3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:10.021445 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:10.047298 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54470ecf-b275-4982-9e34-fa698f2342b3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:10.087864 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26b4995-c54c-44ad-8772-470f92f33005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:10.088736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b4995-c54c-44ad-8772-470f92f33005 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:10.089023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b4995-c54c-44ad-8772-470f92f33005 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:10.089305 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26b4995-c54c-44ad-8772-470f92f33005 None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:10.090550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b4995-c54c-44ad-8772-470f92f33005 None admin] RBAC: Authorizing `identity:get_project(project_id=40f51bfd8db04611b068f7edd7002004)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:10.092040 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26b4995-c54c-44ad-8772-470f92f33005 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:10.094074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 74/155] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:09 2022] GET /identity/v3/projects/40f51bfd8db04611b068f7edd7002004 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:10.100250 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274ac3c-5b5d-451c-9890-4d9fa137c681 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:10.152763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:10.205880 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274ac3c-5b5d-451c-9890-4d9fa137c681 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:10.242688 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ef696-e8c1-47ec-aeb2-469f0b75696a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:10.243673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ef696-e8c1-47ec-aeb2-469f0b75696a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:10.243959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ef696-e8c1-47ec-aeb2-469f0b75696a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:10.244239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ef696-e8c1-47ec-aeb2-469f0b75696a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:10.245089 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ef696-e8c1-47ec-aeb2-469f0b75696a None admin] RBAC: Adding query filter params (role.id=d7aadc20cd4c4838ae5d7d8e8826abae, scope.project.id=40f51bfd8db04611b068f7edd7002004, user.id=1b3131ee9b304a72b169fab71c8af3d5) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:10.245405 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ef696-e8c1-47ec-aeb2-469f0b75696a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:10.246903 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ef696-e8c1-47ec-aeb2-469f0b75696a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:10.252162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbf871-17b2-44b0-be1b-7db178abb4c1 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:10.252452 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbf871-17b2-44b0-be1b-7db178abb4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:10.252736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cbf871-17b2-44b0-be1b-7db178abb4c1 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:10.254124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 75/156] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:10.258227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 82/157] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:10 2022] GET /identity/v3/role_assignments?user.id=1b3131ee9b304a72b169fab71c8af3d5&scope.project.id=40f51bfd8db04611b068f7edd7002004&role.id=d7aadc20cd4c4838ae5d7d8e8826abae => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:10.264411 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc0e26-3dd7-4318-81d5-b12b5c0f81cf None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:10.264791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc0e26-3dd7-4318-81d5-b12b5c0f81cf None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:10.265030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc0e26-3dd7-4318-81d5-b12b5c0f81cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:10.287741 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-74bc0e26-3dd7-4318-81d5-b12b5c0f81cf None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:10.288963 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-74bc0e26-3dd7-4318-81d5-b12b5c0f81cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:10.375353 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 76/158] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:10.419820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ef696-e8c1-47ec-aeb2-469f0b75696a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:10.535214 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ef696-e8c1-47ec-aeb2-469f0b75696a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:10.561050 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:10.579091 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11552b74-6c22-48d2-9702-1f0085033266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:10.579942 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11552b74-6c22-48d2-9702-1f0085033266 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:10.580451 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11552b74-6c22-48d2-9702-1f0085033266 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:10.580541 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11552b74-6c22-48d2-9702-1f0085033266 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:10.591769 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11552b74-6c22-48d2-9702-1f0085033266 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:10.600148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-11552b74-6c22-48d2-9702-1f0085033266 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:58:10.600148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:10.600148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:10.600148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:10.600148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:10.600148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:10.600148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:10.600148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:10.600148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:10.600148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:10.600148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:10.600148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:10.600148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:10.600148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:10.600148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:10.600148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:10.601086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:10.601086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:10.601086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:10.601086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:10.601086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:10.601086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:10.601086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:10.601086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:10.601086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:10.601086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:10.601086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:10.601086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:10.601086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:10.601086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:10.601086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:10.601086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:10.601938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:10.601938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:10.601938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:10.601938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:10.601938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:10.601938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:10.601938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:10.601938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:10.601938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:10.601938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:10.601938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:58:10.601938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:10.608195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 83/159] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:58:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:10.614345 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc0e26-3dd7-4318-81d5-b12b5c0f81cf None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:10.737047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:10.755492 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc0e26-3dd7-4318-81d5-b12b5c0f81cf None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:10.798864 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c24e4c-3c13-42f8-9f3e-94b1fa3d5111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:10.799541 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c24e4c-3c13-42f8-9f3e-94b1fa3d5111 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:10.799827 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c24e4c-3c13-42f8-9f3e-94b1fa3d5111 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:10.800107 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c24e4c-3c13-42f8-9f3e-94b1fa3d5111 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:10.800869 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c24e4c-3c13-42f8-9f3e-94b1fa3d5111 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:10.801181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c24e4c-3c13-42f8-9f3e-94b1fa3d5111 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:10.803198 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c24e4c-3c13-42f8-9f3e-94b1fa3d5111 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:10.812359 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 77/160] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:10.819687 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11552b74-6c22-48d2-9702-1f0085033266 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:10.957984 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11552b74-6c22-48d2-9702-1f0085033266 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:10.997926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec16af71-e3cb-444a-8c9d-8c67b9b27c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:10.999278 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16af71-e3cb-444a-8c9d-8c67b9b27c50 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:10.999762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16af71-e3cb-444a-8c9d-8c67b9b27c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:11.000257 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16af71-e3cb-444a-8c9d-8c67b9b27c50 None admin] PATH_INFO: `/v3/users/c531d2732de94f47a66f16db7047917f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:11.001674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16af71-e3cb-444a-8c9d-8c67b9b27c50 None admin] RBAC: Authorizing `identity:get_user(user_id=c531d2732de94f47a66f16db7047917f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:11.003458 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec16af71-e3cb-444a-8c9d-8c67b9b27c50 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:11.005510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 84/161] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:10 2022] GET /identity/v3/users/c531d2732de94f47a66f16db7047917f => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:11.011392 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c24e4c-3c13-42f8-9f3e-94b1fa3d5111 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:11.148723 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c24e4c-3c13-42f8-9f3e-94b1fa3d5111 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:11.189894 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79239114-ec2a-45d0-b5bd-b4619e6f9aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:11.190576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79239114-ec2a-45d0-b5bd-b4619e6f9aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:11.190866 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79239114-ec2a-45d0-b5bd-b4619e6f9aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:11.191149 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79239114-ec2a-45d0-b5bd-b4619e6f9aa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:11.220261 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:11.271451 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79239114-ec2a-45d0-b5bd-b4619e6f9aa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:11.272928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79239114-ec2a-45d0-b5bd-b4619e6f9aa7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:11.395583 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce8328-1a46-4c93-9111-edb0e29fbe43 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:11.396243 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce8328-1a46-4c93-9111-edb0e29fbe43 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:11.396739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce8328-1a46-4c93-9111-edb0e29fbe43 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:11.398334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 85/162] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:11.399372 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 78/163] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:11.404375 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79239114-ec2a-45d0-b5bd-b4619e6f9aa7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:11.404656 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea7a12-6b16-4353-a7cf-3f494097f343 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:11.405160 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea7a12-6b16-4353-a7cf-3f494097f343 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:11.405641 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea7a12-6b16-4353-a7cf-3f494097f343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:11.423904 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-feea7a12-6b16-4353-a7cf-3f494097f343 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:11.425542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-feea7a12-6b16-4353-a7cf-3f494097f343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:11.462106 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:11.467059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:11.508637 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 86/164] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:11.517032 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a52ed-4401-44b9-bfb0-9e15026cce86 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:11.517573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a52ed-4401-44b9-bfb0-9e15026cce86 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:11.518051 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a52ed-4401-44b9-bfb0-9e15026cce86 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:11.519718 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 87/165] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:11.525535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fcb6a-d296-4750-ae5e-028527c5b497 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:11.526051 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fcb6a-d296-4750-ae5e-028527c5b497 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:11.526692 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fcb6a-d296-4750-ae5e-028527c5b497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:11.530063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79239114-ec2a-45d0-b5bd-b4619e6f9aa7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:11.550658 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4d1fcb6a-d296-4750-ae5e-028527c5b497 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:11.552399 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4d1fcb6a-d296-4750-ae5e-028527c5b497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:11.573200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07150471-004a-4054-a7db-64da0b7539d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:11.574062 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07150471-004a-4054-a7db-64da0b7539d2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:11.574346 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07150471-004a-4054-a7db-64da0b7539d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:11.574645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07150471-004a-4054-a7db-64da0b7539d2 None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:11.575994 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07150471-004a-4054-a7db-64da0b7539d2 None admin] RBAC: Authorizing `identity:get_project(project_id=40f51bfd8db04611b068f7edd7002004)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:11.577447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07150471-004a-4054-a7db-64da0b7539d2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:11.579467 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 79/166] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:11 2022] GET /identity/v3/projects/40f51bfd8db04611b068f7edd7002004 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:11.580360 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07150471-004a-4054-a7db-64da0b7539d2 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:11.642782 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 88/167] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:58:11.644633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fcb6a-d296-4750-ae5e-028527c5b497 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:11.682891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07150471-004a-4054-a7db-64da0b7539d2 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:11.697935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:11.721805 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1462642-1954-454e-8029-89176aaa3195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:11.722645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1462642-1954-454e-8029-89176aaa3195 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:11.722977 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1462642-1954-454e-8029-89176aaa3195 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:11.723679 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1462642-1954-454e-8029-89176aaa3195 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:11.733879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1462642-1954-454e-8029-89176aaa3195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:11.742766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e1462642-1954-454e-8029-89176aaa3195 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:11.742766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:11.742766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:11.742766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:11.742766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:11.742766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:11.742766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:11.742766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:11.742766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:11.742766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:11.742766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:11.742766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:11.742766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:11.742766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:11.742766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:11.742766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:11.744265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:11.744265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:11.744265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:11.744265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:11.744265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:11.744265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:11.744265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:11.744265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:11.744265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:11.744265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:11.744265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:11.744265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:11.744265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:11.744265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:11.744265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:11.744265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:11.745312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:11.745312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:11.745312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:11.745312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:11.745312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:11.745312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:11.745312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:11.745312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:11.745312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:11.745312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:11.745312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:11.745312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:11.747701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 80/168] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:58:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:11.751324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1462642-1954-454e-8029-89176aaa3195 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:11.769707 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fcb6a-d296-4750-ae5e-028527c5b497 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:11.772502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:11.810409 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a5d10-a633-4d81-b518-927a1962a366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:11.811570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a5d10-a633-4d81-b518-927a1962a366 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:11.812120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a5d10-a633-4d81-b518-927a1962a366 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:11.812606 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a5d10-a633-4d81-b518-927a1962a366 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:11.813653 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a5d10-a633-4d81-b518-927a1962a366 None admin] RBAC: Adding query filter params (role.id=f0f7e28cbe314921afdfb657b163436f, scope.project.id=40f51bfd8db04611b068f7edd7002004, user.id=c531d2732de94f47a66f16db7047917f) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:11.814297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a5d10-a633-4d81-b518-927a1962a366 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:11.816091 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a5d10-a633-4d81-b518-927a1962a366 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:11.823848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 89/169] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:11 2022] GET /identity/v3/role_assignments?user.id=c531d2732de94f47a66f16db7047917f&scope.project.id=40f51bfd8db04611b068f7edd7002004&role.id=f0f7e28cbe314921afdfb657b163436f => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:11.825103 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a5d10-a633-4d81-b518-927a1962a366 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:11.859899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1462642-1954-454e-8029-89176aaa3195 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:11.906568 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a73f6-96ea-4bb1-a634-e066bb6bd806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:11.907376 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a73f6-96ea-4bb1-a634-e066bb6bd806 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:11.907665 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a73f6-96ea-4bb1-a634-e066bb6bd806 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:11.907945 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a73f6-96ea-4bb1-a634-e066bb6bd806 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:11.932198 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a73f6-96ea-4bb1-a634-e066bb6bd806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:11.949369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-0c6a73f6-96ea-4bb1-a634-e066bb6bd806 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:11.949369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:11.949369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:11.949369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:11.949369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:11.949369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:11.949369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:11.949369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:11.949369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:11.949369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:11.949369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:11.949369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:11.949369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:11.949369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:11.949369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:11.949369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:11.950424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:11.950424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:11.950424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:11.950424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:11.950424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:11.950424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:11.950424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:11.950424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:11.950424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:11.950424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:11.950424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:11.950424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:11.950424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:11.950424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:11.950424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:11.950424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:11.951234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:11.951234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:11.951234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:11.951234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:11.951234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:11.951234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:11.951234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:11.951234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:11.951234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:11.951234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:11.951234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:11.951234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:11.951234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 81/170] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:11.961111 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a73f6-96ea-4bb1-a634-e066bb6bd806 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:12.020383 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a5d10-a633-4d81-b518-927a1962a366 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:12.022794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:12.064293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae531fe2-381a-418e-8fc2-8043e18bc681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:12.065238 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae531fe2-381a-418e-8fc2-8043e18bc681 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:12.065631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae531fe2-381a-418e-8fc2-8043e18bc681 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:12.066109 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae531fe2-381a-418e-8fc2-8043e18bc681 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:12.067815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae531fe2-381a-418e-8fc2-8043e18bc681 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:12.068251 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae531fe2-381a-418e-8fc2-8043e18bc681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:12.069766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae531fe2-381a-418e-8fc2-8043e18bc681 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:12.084764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 90/171] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:58:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:12.086236 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae531fe2-381a-418e-8fc2-8043e18bc681 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:12.108836 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a73f6-96ea-4bb1-a634-e066bb6bd806 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:12.120202 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:12.146063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9444f-9a80-4a62-a289-1eb830673d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:12.147050 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9444f-9a80-4a62-a289-1eb830673d9e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:12.147511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9444f-9a80-4a62-a289-1eb830673d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:12.147930 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9444f-9a80-4a62-a289-1eb830673d9e None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:12.148891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9444f-9a80-4a62-a289-1eb830673d9e None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:12.149343 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9444f-9a80-4a62-a289-1eb830673d9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:12.150984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9444f-9a80-4a62-a289-1eb830673d9e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:12.160007 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 82/172] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:12.166163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9444f-9a80-4a62-a289-1eb830673d9e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:12.207770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae531fe2-381a-418e-8fc2-8043e18bc681 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:12.249674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace66e7a-ed74-4e22-96ec-ab610a91fb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:12.250947 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace66e7a-ed74-4e22-96ec-ab610a91fb6a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:12.251348 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace66e7a-ed74-4e22-96ec-ab610a91fb6a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:12.251739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace66e7a-ed74-4e22-96ec-ab610a91fb6a None admin] PATH_INFO: `/v3/users/1b3131ee9b304a72b169fab71c8af3d5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:12.253155 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace66e7a-ed74-4e22-96ec-ab610a91fb6a None admin] RBAC: Authorizing `identity:get_user(user_id=1b3131ee9b304a72b169fab71c8af3d5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:12.254991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace66e7a-ed74-4e22-96ec-ab610a91fb6a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:12.260608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 91/173] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:12 2022] GET /identity/v3/users/1b3131ee9b304a72b169fab71c8af3d5 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:12.263223 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace66e7a-ed74-4e22-96ec-ab610a91fb6a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:12.276149 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:12.282803 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9444f-9a80-4a62-a289-1eb830673d9e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:12.320467 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6888ef-1cae-4bfc-8038-77acb0834956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:12.321634 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6888ef-1cae-4bfc-8038-77acb0834956 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:12.322119 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6888ef-1cae-4bfc-8038-77acb0834956 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:12.322566 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6888ef-1cae-4bfc-8038-77acb0834956 None admin] PATH_INFO: `/v3/users/1b3131ee9b304a72b169fab71c8af3d5` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:12.323920 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6888ef-1cae-4bfc-8038-77acb0834956 None admin] RBAC: Authorizing `identity:get_user(user_id=1b3131ee9b304a72b169fab71c8af3d5)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:12.325520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6888ef-1cae-4bfc-8038-77acb0834956 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:12.327409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 83/174] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:12 2022] GET /identity/v3/users/1b3131ee9b304a72b169fab71c8af3d5 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:12.333259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6888ef-1cae-4bfc-8038-77acb0834956 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:12.374806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:12.382169 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace66e7a-ed74-4e22-96ec-ab610a91fb6a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:12.423002 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cf0bb-3550-4843-9adc-bc14504a5658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:12.423678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cf0bb-3550-4843-9adc-bc14504a5658 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:12.423968 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cf0bb-3550-4843-9adc-bc14504a5658 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:12.424252 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cf0bb-3550-4843-9adc-bc14504a5658 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:12.443371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6888ef-1cae-4bfc-8038-77acb0834956 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:12.480685 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f4c1f-3551-4eac-8429-3dc8982efa73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:12.481672 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f4c1f-3551-4eac-8429-3dc8982efa73 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:12.482191 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f4c1f-3551-4eac-8429-3dc8982efa73 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:12.482711 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f4c1f-3551-4eac-8429-3dc8982efa73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:12.499740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6cf0bb-3550-4843-9adc-bc14504a5658 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:12.501426 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6cf0bb-3550-4843-9adc-bc14504a5658 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:12.517540 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:12.563999 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f4c1f-3551-4eac-8429-3dc8982efa73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:12.565483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2f4c1f-3551-4eac-8429-3dc8982efa73 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:12.604630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:12.612141 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 92/175] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 03:58:12.619032 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cf0bb-3550-4843-9adc-bc14504a5658 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:12.727701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cf0bb-3550-4843-9adc-bc14504a5658 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:12.769656 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d00a1d-ac83-4c7f-a79d-824c775cd1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:12.770521 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d00a1d-ac83-4c7f-a79d-824c775cd1a4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:12.770816 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d00a1d-ac83-4c7f-a79d-824c775cd1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:12.771086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d00a1d-ac83-4c7f-a79d-824c775cd1a4 None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:12.772607 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d00a1d-ac83-4c7f-a79d-824c775cd1a4 None admin] RBAC: Authorizing `identity:get_project(project_id=40f51bfd8db04611b068f7edd7002004)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:12.774098 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d00a1d-ac83-4c7f-a79d-824c775cd1a4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:12.776032 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 93/176] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:12 2022] GET /identity/v3/projects/40f51bfd8db04611b068f7edd7002004 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:12.781902 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d00a1d-ac83-4c7f-a79d-824c775cd1a4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:12.815099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 84/177] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 03:58:12.821592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f4c1f-3551-4eac-8429-3dc8982efa73 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:12.850043 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:12.893989 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d00a1d-ac83-4c7f-a79d-824c775cd1a4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:12.931897 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad012e1-e5eb-44fd-8d88-950297c53f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:12.932589 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad012e1-e5eb-44fd-8d88-950297c53f40 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:12.932863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad012e1-e5eb-44fd-8d88-950297c53f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:12.933151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad012e1-e5eb-44fd-8d88-950297c53f40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:12.934003 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad012e1-e5eb-44fd-8d88-950297c53f40 None admin] RBAC: Adding query filter params (role.id=a048041c27f44edf98a110d23138a327, scope.project.id=40f51bfd8db04611b068f7edd7002004, user.id=1b3131ee9b304a72b169fab71c8af3d5) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:12.934321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad012e1-e5eb-44fd-8d88-950297c53f40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:12.935802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad012e1-e5eb-44fd-8d88-950297c53f40 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:12.942490 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 94/178] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:12 2022] GET /identity/v3/role_assignments?user.id=1b3131ee9b304a72b169fab71c8af3d5&scope.project.id=40f51bfd8db04611b068f7edd7002004&role.id=a048041c27f44edf98a110d23138a327 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:12.955582 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:13.005472 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2f4c1f-3551-4eac-8429-3dc8982efa73 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:13.108100 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3082b2f-e047-473e-9855-b9d19b5f4578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:13.109224 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3082b2f-e047-473e-9855-b9d19b5f4578 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:13.109683 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3082b2f-e047-473e-9855-b9d19b5f4578 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:13.110104 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3082b2f-e047-473e-9855-b9d19b5f4578 None admin] PATH_INFO: `/v3/projects/130b568f2ae44a118343de8697aec423` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:13.111530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3082b2f-e047-473e-9855-b9d19b5f4578 None admin] RBAC: Authorizing `identity:get_project(project_id=130b568f2ae44a118343de8697aec423)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:13.113158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3082b2f-e047-473e-9855-b9d19b5f4578 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:13.115632 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 85/179] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:12 2022] GET /identity/v3/projects/130b568f2ae44a118343de8697aec423 => generated 312 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:13.122001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad012e1-e5eb-44fd-8d88-950297c53f40 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:13.248494 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad012e1-e5eb-44fd-8d88-950297c53f40 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:13.257865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42303f56-a8fa-4b23-abd1-be68247fbc57 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:13.258518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42303f56-a8fa-4b23-abd1-be68247fbc57 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:13.259010 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42303f56-a8fa-4b23-abd1-be68247fbc57 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:13.261837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 86/180] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:13.271280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa3142-bde4-4eef-a658-3555cb18a4d1 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:13.271574 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa3142-bde4-4eef-a658-3555cb18a4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:13.271861 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa3142-bde4-4eef-a658-3555cb18a4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:13.292799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-dfaa3142-bde4-4eef-a658-3555cb18a4d1 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:13.294338 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dfaa3142-bde4-4eef-a658-3555cb18a4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:13.297160 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:13.315672 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f85d9-731a-45c9-8539-4fcf82fe2c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:13.316353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f85d9-731a-45c9-8539-4fcf82fe2c55 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:13.316655 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f85d9-731a-45c9-8539-4fcf82fe2c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:13.317034 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f85d9-731a-45c9-8539-4fcf82fe2c55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:13.317897 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f85d9-731a-45c9-8539-4fcf82fe2c55 None admin] RBAC: Adding query filter params (role.id=d7aadc20cd4c4838ae5d7d8e8826abae, scope.project.id=130b568f2ae44a118343de8697aec423, user.id=1b3131ee9b304a72b169fab71c8af3d5) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:13.318219 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f85d9-731a-45c9-8539-4fcf82fe2c55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:13.319721 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f85d9-731a-45c9-8539-4fcf82fe2c55 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:13.332959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 95/181] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:13 2022] GET /identity/v3/role_assignments?user.id=1b3131ee9b304a72b169fab71c8af3d5&scope.project.id=130b568f2ae44a118343de8697aec423&role.id=d7aadc20cd4c4838ae5d7d8e8826abae => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:13.389893 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 87/182] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:13.428372 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f85d9-731a-45c9-8539-4fcf82fe2c55 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:13.458594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84881be0-8da0-4989-bb07-5576f2bdf212 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:13.458594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84881be0-8da0-4989-bb07-5576f2bdf212 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:13.458594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84881be0-8da0-4989-bb07-5576f2bdf212 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:13.460206 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 88/183] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:13.467175 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d8469-846f-4eb9-b257-49d9ecd56a1d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:13.467503 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d8469-846f-4eb9-b257-49d9ecd56a1d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:13.467775 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d8469-846f-4eb9-b257-49d9ecd56a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:13.485470 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d02d8469-846f-4eb9-b257-49d9ecd56a1d None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:13.486716 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d02d8469-846f-4eb9-b257-49d9ecd56a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:13.563048 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:13.574932 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 89/184] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:58:13.588330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f85d9-731a-45c9-8539-4fcf82fe2c55 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:13.601062 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d8469-846f-4eb9-b257-49d9ecd56a1d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:13.627814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d487507-2e8d-44d9-93d6-aa1d861c9336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:13.629114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d487507-2e8d-44d9-93d6-aa1d861c9336 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:13.629749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d487507-2e8d-44d9-93d6-aa1d861c9336 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:13.630319 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d487507-2e8d-44d9-93d6-aa1d861c9336 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:13.645565 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d487507-2e8d-44d9-93d6-aa1d861c9336 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:13.654179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-3d487507-2e8d-44d9-93d6-aa1d861c9336 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:58:13.654179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:13.654179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:13.654179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:13.654179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:13.654179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:13.654179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:13.654179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:13.654179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:13.654179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:13.654179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:13.654179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:13.654179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:13.654179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:13.654179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:13.654179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:13.655123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:13.655123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:13.655123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:13.655123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:13.655123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:13.655123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:13.655123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:13.655123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:13.655123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:13.655123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:13.655123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:13.655123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:13.655123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:13.655123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:13.655123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:13.655123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:13.655926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:13.655926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:13.655926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:13.655926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:13.655926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:13.655926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:13.655926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:13.655926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:13.655926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:13.655926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:13.655926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:58:13.655926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:13.659893 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 96/185] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:58:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:13.666779 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d487507-2e8d-44d9-93d6-aa1d861c9336 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:13.707568 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d8469-846f-4eb9-b257-49d9ecd56a1d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:13.727113 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:13.750447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b064ba-5460-4289-b5ff-c699081b8acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:13.751428 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b064ba-5460-4289-b5ff-c699081b8acf None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:13.751719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b064ba-5460-4289-b5ff-c699081b8acf None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:13.752785 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b064ba-5460-4289-b5ff-c699081b8acf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:13.764621 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b064ba-5460-4289-b5ff-c699081b8acf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:13.772845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-f8b064ba-5460-4289-b5ff-c699081b8acf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:13.772845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:13.772845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:13.772845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:13.772845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:13.772845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:13.772845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:13.772845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:13.772845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:13.772845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:13.772845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:13.772845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:13.772845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:13.772845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:13.772845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:13.772845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:13.773726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:13.773726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:13.773726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:13.773726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:13.773726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:13.773726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:13.773726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:13.773726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:13.773726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:13.773726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:13.773726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:13.773726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:13.773726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:13.773726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:13.773726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:13.773726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:13.774610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:13.774610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:13.774610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:13.774610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:13.774610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:13.774610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:13.774610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:13.774610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:13.774610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:13.774610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:13.774610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:13.774610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:13.774610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 90/186] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:58:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:58:13.775334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9463855-045f-433d-b2bc-c027857faacd None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:13.776847 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9463855-045f-433d-b2bc-c027857faacd None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:13.776847 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9463855-045f-433d-b2bc-c027857faacd None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:13.777304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 91/187] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:13.778209 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29edbc-cbc8-4c0d-9fc2-30ab8bb61731 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:13.780779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29edbc-cbc8-4c0d-9fc2-30ab8bb61731 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:13.780779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29edbc-cbc8-4c0d-9fc2-30ab8bb61731 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:13.780779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 92/188] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:13.783612 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2de078-a70d-46b8-80ee-1eaa5495245e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:13.783899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2de078-a70d-46b8-80ee-1eaa5495245e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:13.784177 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2de078-a70d-46b8-80ee-1eaa5495245e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:13.794770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d487507-2e8d-44d9-93d6-aa1d861c9336 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:13.801132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0a2de078-a70d-46b8-80ee-1eaa5495245e None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:13.803137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0a2de078-a70d-46b8-80ee-1eaa5495245e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:13.829083 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf06e82-1f50-489f-a584-9afce0f52d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:13.829689 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf06e82-1f50-489f-a584-9afce0f52d5c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:13.829973 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf06e82-1f50-489f-a584-9afce0f52d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:13.830250 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf06e82-1f50-489f-a584-9afce0f52d5c None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:13.831029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf06e82-1f50-489f-a584-9afce0f52d5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:13.831339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf06e82-1f50-489f-a584-9afce0f52d5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:13.833286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf06e82-1f50-489f-a584-9afce0f52d5c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:13.841210 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 97/189] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:13.841992 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf06e82-1f50-489f-a584-9afce0f52d5c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:13.850297 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:13.870794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 93/190] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:58:13.872125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37857e5-1061-4c2d-9962-25c47e1efea5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:13.872421 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37857e5-1061-4c2d-9962-25c47e1efea5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:13.872703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37857e5-1061-4c2d-9962-25c47e1efea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:13.888853 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e37857e5-1061-4c2d-9962-25c47e1efea5 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:13.890048 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e37857e5-1061-4c2d-9962-25c47e1efea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:13.951212 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf06e82-1f50-489f-a584-9afce0f52d5c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:13.958651 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 94/191] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:13.959496 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37857e5-1061-4c2d-9962-25c47e1efea5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:13.972686 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:13.995415 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8e0e5-b489-4e10-8632-ea21d5697c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:13.996080 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8e0e5-b489-4e10-8632-ea21d5697c15 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:13.996363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8e0e5-b489-4e10-8632-ea21d5697c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:13.996639 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8e0e5-b489-4e10-8632-ea21d5697c15 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:13.997391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8e0e5-b489-4e10-8632-ea21d5697c15 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:13.997706 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8e0e5-b489-4e10-8632-ea21d5697c15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:13.999126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8e0e5-b489-4e10-8632-ea21d5697c15 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:14.008610 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 98/192] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:58:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:14.013784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8e0e5-b489-4e10-8632-ea21d5697c15 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:14.064569 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37857e5-1061-4c2d-9962-25c47e1efea5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:14.077597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:14.098701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8811842-f5b7-41b1-afc9-ae9731d81229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:14.099999 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8811842-f5b7-41b1-afc9-ae9731d81229 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:14.100282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8811842-f5b7-41b1-afc9-ae9731d81229 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:14.100612 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8811842-f5b7-41b1-afc9-ae9731d81229 None admin] PATH_INFO: `/v3/users/381eba1a68b54e2eabaf4f64a95e41cd` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:14.113693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8e0e5-b489-4e10-8632-ea21d5697c15 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:14.115224 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8811842-f5b7-41b1-afc9-ae9731d81229 None admin] RBAC: Authorizing `identity:get_user(user_id=381eba1a68b54e2eabaf4f64a95e41cd)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:14.116703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8811842-f5b7-41b1-afc9-ae9731d81229 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:14.118640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 95/193] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:13 2022] GET /identity/v3/users/381eba1a68b54e2eabaf4f64a95e41cd => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:14.119407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8811842-f5b7-41b1-afc9-ae9731d81229 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:14.148779 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3405e-713b-4d4e-85ed-b15fb7e34db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:14.149578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3405e-713b-4d4e-85ed-b15fb7e34db6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:14.149871 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3405e-713b-4d4e-85ed-b15fb7e34db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:14.150152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3405e-713b-4d4e-85ed-b15fb7e34db6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:14.160451 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3405e-713b-4d4e-85ed-b15fb7e34db6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:14.168791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-04c3405e-713b-4d4e-85ed-b15fb7e34db6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 03:58:14.168791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:14.168791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:14.168791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:14.168791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:14.168791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:14.168791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:14.168791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:14.168791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:14.168791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:14.168791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:14.168791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:14.168791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:14.168791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:14.168791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:14.168791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:14.169845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:14.169845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:14.169845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:14.169845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:14.169845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:14.169845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:14.169845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:14.169845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:14.169845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:14.169845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:14.169845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:14.169845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:14.169845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:14.169845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:14.169845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:14.169845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:14.170762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:14.170762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:14.170762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:14.170762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:14.170762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:14.170762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:14.170762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:14.170762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:14.170762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:14.170762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:14.170762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 03:58:14.170762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:14.170762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 99/194] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:14.170762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3405e-713b-4d4e-85ed-b15fb7e34db6 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:14.204688 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:14.222948 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8811842-f5b7-41b1-afc9-ae9731d81229 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:14.259264 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd348581-6219-433c-9364-a2182dd56594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:14.260137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd348581-6219-433c-9364-a2182dd56594 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:14.260425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd348581-6219-433c-9364-a2182dd56594 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:14.260701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd348581-6219-433c-9364-a2182dd56594 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:14.261879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd348581-6219-433c-9364-a2182dd56594 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:14.263721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd348581-6219-433c-9364-a2182dd56594 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:14.266991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 96/195] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:14.268169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd348581-6219-433c-9364-a2182dd56594 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:14.274299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3405e-713b-4d4e-85ed-b15fb7e34db6 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:14.310157 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca058bb-f9ce-4686-aff5-36e94592498c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:14.310984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:14.311109 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca058bb-f9ce-4686-aff5-36e94592498c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:14.311354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca058bb-f9ce-4686-aff5-36e94592498c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:14.311634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca058bb-f9ce-4686-aff5-36e94592498c None admin] PATH_INFO: `/v3/users/381eba1a68b54e2eabaf4f64a95e41cd` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:14.312885 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca058bb-f9ce-4686-aff5-36e94592498c None admin] RBAC: Authorizing `identity:get_user(user_id=381eba1a68b54e2eabaf4f64a95e41cd)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:14.314358 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca058bb-f9ce-4686-aff5-36e94592498c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:14.316027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 100/196] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:14 2022] GET /identity/v3/users/381eba1a68b54e2eabaf4f64a95e41cd => generated 283 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:14.317588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca058bb-f9ce-4686-aff5-36e94592498c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:14.378224 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd348581-6219-433c-9364-a2182dd56594 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:14.412930 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9adbc-44aa-4f2e-87ce-ef6a104fb321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:14.413575 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9adbc-44aa-4f2e-87ce-ef6a104fb321 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:14.413855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9adbc-44aa-4f2e-87ce-ef6a104fb321 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:14.414132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9adbc-44aa-4f2e-87ce-ef6a104fb321 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:14.420741 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca058bb-f9ce-4686-aff5-36e94592498c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:14.423117 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:14.456207 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dcaf6f-13fc-4bf2-84d6-34c4803de67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:14.457159 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcaf6f-13fc-4bf2-84d6-34c4803de67c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:14.458037 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcaf6f-13fc-4bf2-84d6-34c4803de67c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:14.458393 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcaf6f-13fc-4bf2-84d6-34c4803de67c None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:14.459162 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dcaf6f-13fc-4bf2-84d6-34c4803de67c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:14.459479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dcaf6f-13fc-4bf2-84d6-34c4803de67c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:14.460959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dcaf6f-13fc-4bf2-84d6-34c4803de67c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:14.469763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 101/197] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:14.474301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dcaf6f-13fc-4bf2-84d6-34c4803de67c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:14.483005 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9adbc-44aa-4f2e-87ce-ef6a104fb321 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:14.484472 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9adbc-44aa-4f2e-87ce-ef6a104fb321 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:14.522903 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:14.569923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dcaf6f-13fc-4bf2-84d6-34c4803de67c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:14.583653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 97/198] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:14.584492 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9adbc-44aa-4f2e-87ce-ef6a104fb321 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:14.607909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234da631-3856-4a85-b722-ec46249bbd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:14.608586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234da631-3856-4a85-b722-ec46249bbd56 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:14.608875 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234da631-3856-4a85-b722-ec46249bbd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:14.609158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234da631-3856-4a85-b722-ec46249bbd56 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:14.609875 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234da631-3856-4a85-b722-ec46249bbd56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:14.611395 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234da631-3856-4a85-b722-ec46249bbd56 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:14.681049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9adbc-44aa-4f2e-87ce-ef6a104fb321 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:14.716200 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 102/199] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jun 24 03:58:14 2022] POST /identity/v3/users => generated 297 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:58:14.716756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a2255-2ef9-462a-b58b-e29b5e783850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:14.717038 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234da631-3856-4a85-b722-ec46249bbd56 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:14.717483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a2255-2ef9-462a-b58b-e29b5e783850 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:14.717715 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a2255-2ef9-462a-b58b-e29b5e783850 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:14.717961 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7a2255-2ef9-462a-b58b-e29b5e783850 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:14.734524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:14.748778 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:14.811215 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a2255-2ef9-462a-b58b-e29b5e783850 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:14.812730 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7a2255-2ef9-462a-b58b-e29b5e783850 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:14.833742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234da631-3856-4a85-b722-ec46249bbd56 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:14.872368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad80a0-7187-444c-a9b8-02337ff9d2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:14.873266 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad80a0-7187-444c-a9b8-02337ff9d2dc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:14.873553 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad80a0-7187-444c-a9b8-02337ff9d2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:14.873838 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad80a0-7187-444c-a9b8-02337ff9d2dc None admin] PATH_INFO: `/v3/users/e22bb34e90cd4ea498ea37049b75163d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:14.875102 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad80a0-7187-444c-a9b8-02337ff9d2dc None admin] RBAC: Authorizing `identity:get_user(user_id=e22bb34e90cd4ea498ea37049b75163d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:14.876601 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad80a0-7187-444c-a9b8-02337ff9d2dc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:14.878913 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 103/200] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:14 2022] GET /identity/v3/users/e22bb34e90cd4ea498ea37049b75163d => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:14.880370 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad80a0-7187-444c-a9b8-02337ff9d2dc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:14.934108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 98/201] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:14.934861 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a2255-2ef9-462a-b58b-e29b5e783850 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:14.980892 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:14.991757 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:15.010524 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad80a0-7187-444c-a9b8-02337ff9d2dc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:15.064523 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd00186-d17a-4e3b-8f1f-6c778aea55c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:15.065057 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd00186-d17a-4e3b-8f1f-6c778aea55c1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:15.065349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd00186-d17a-4e3b-8f1f-6c778aea55c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:15.065632 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd00186-d17a-4e3b-8f1f-6c778aea55c1 None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:15.066067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7a2255-2ef9-462a-b58b-e29b5e783850 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:15.067629 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd00186-d17a-4e3b-8f1f-6c778aea55c1 None admin] RBAC: Authorizing `identity:get_project(project_id=e87c7eebde3941f1aa5f40ce794e8687)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:15.070573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd00186-d17a-4e3b-8f1f-6c778aea55c1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:15.075892 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 104/202] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:14 2022] GET /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:15.075892 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd00186-d17a-4e3b-8f1f-6c778aea55c1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:15.113216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa6057-80c0-41f8-9a08-ab964c10fe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:15.114570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa6057-80c0-41f8-9a08-ab964c10fe13 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:15.114861 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa6057-80c0-41f8-9a08-ab964c10fe13 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:15.115183 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa6057-80c0-41f8-9a08-ab964c10fe13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:15.192935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd00186-d17a-4e3b-8f1f-6c778aea55c1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:15.212787 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa6057-80c0-41f8-9a08-ab964c10fe13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:15.215170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa6057-80c0-41f8-9a08-ab964c10fe13 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:15.239561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871c903-4a86-4c59-9158-94d9f0221c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:15.240927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871c903-4a86-4c59-9158-94d9f0221c87 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:15.241520 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871c903-4a86-4c59-9158-94d9f0221c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:15.241972 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871c903-4a86-4c59-9158-94d9f0221c87 None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:15.244241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871c903-4a86-4c59-9158-94d9f0221c87 None admin] RBAC: Authorizing `identity:get_project(project_id=e87c7eebde3941f1aa5f40ce794e8687)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:15.246318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871c903-4a86-4c59-9158-94d9f0221c87 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:15.248672 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 105/203] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:15 2022] GET /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:15.249570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871c903-4a86-4c59-9158-94d9f0221c87 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:15.263850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:15.285595 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:15.341198 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 99/204] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:15.342279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa6057-80c0-41f8-9a08-ab964c10fe13 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:15.373678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871c903-4a86-4c59-9158-94d9f0221c87 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:15.411517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520dab5-3e38-4f1a-8b07-3915442affd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:15.412458 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520dab5-3e38-4f1a-8b07-3915442affd8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:15.412891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520dab5-3e38-4f1a-8b07-3915442affd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:15.413354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0520dab5-3e38-4f1a-8b07-3915442affd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:15.414419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0520dab5-3e38-4f1a-8b07-3915442affd8 None admin] RBAC: Adding query filter params (role.id=f0f7e28cbe314921afdfb657b163436f, scope.project.id=e87c7eebde3941f1aa5f40ce794e8687, user.id=381eba1a68b54e2eabaf4f64a95e41cd) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:15.414844 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0520dab5-3e38-4f1a-8b07-3915442affd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:15.416405 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0520dab5-3e38-4f1a-8b07-3915442affd8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:15.422754 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 106/205] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:15 2022] GET /identity/v3/role_assignments?user.id=381eba1a68b54e2eabaf4f64a95e41cd&scope.project.id=e87c7eebde3941f1aa5f40ce794e8687&role.id=f0f7e28cbe314921afdfb657b163436f => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:15.423694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520dab5-3e38-4f1a-8b07-3915442affd8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:15.465012 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa6057-80c0-41f8-9a08-ab964c10fe13 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:15.504697 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cb8a9-b523-4b37-92f2-598aca12b6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:15.505374 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cb8a9-b523-4b37-92f2-598aca12b6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:15.505661 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cb8a9-b523-4b37-92f2-598aca12b6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:15.505942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cb8a9-b523-4b37-92f2-598aca12b6d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:15.507128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8cb8a9-b523-4b37-92f2-598aca12b6d7 None admin] RBAC: Adding query filter params (role.id=a048041c27f44edf98a110d23138a327, scope.project.id=e87c7eebde3941f1aa5f40ce794e8687, user.id=381eba1a68b54e2eabaf4f64a95e41cd) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:15.507452 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8cb8a9-b523-4b37-92f2-598aca12b6d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:15.508940 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8cb8a9-b523-4b37-92f2-598aca12b6d7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:15.515819 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 100/206] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:15 2022] GET /identity/v3/role_assignments?user.id=381eba1a68b54e2eabaf4f64a95e41cd&scope.project.id=e87c7eebde3941f1aa5f40ce794e8687&role.id=a048041c27f44edf98a110d23138a327 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:15.605774 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:15.623175 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0520dab5-3e38-4f1a-8b07-3915442affd8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:15.671388 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d668efa6-c7cc-48ac-802e-7350bf74908c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:15.672283 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668efa6-c7cc-48ac-802e-7350bf74908c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:15.672574 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668efa6-c7cc-48ac-802e-7350bf74908c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:15.672858 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668efa6-c7cc-48ac-802e-7350bf74908c None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:15.674128 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d668efa6-c7cc-48ac-802e-7350bf74908c None admin] RBAC: Authorizing `identity:get_project(project_id=40f51bfd8db04611b068f7edd7002004)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:15.675669 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d668efa6-c7cc-48ac-802e-7350bf74908c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:15.677653 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 107/207] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:15 2022] GET /identity/v3/projects/40f51bfd8db04611b068f7edd7002004 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:15.683599 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cb8a9-b523-4b37-92f2-598aca12b6d7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:15.687265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:15.797206 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cb8a9-b523-4b37-92f2-598aca12b6d7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:15.831884 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27da007-5deb-4a41-a190-6a41b7ce8ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:15.833418 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27da007-5deb-4a41-a190-6a41b7ce8ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:15.833887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27da007-5deb-4a41-a190-6a41b7ce8ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:15.834345 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27da007-5deb-4a41-a190-6a41b7ce8ef7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:15.835384 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27da007-5deb-4a41-a190-6a41b7ce8ef7 None admin] RBAC: Adding query filter params (role.id=368ae945e96746faa1dc2ecbcefcf0f1, scope.project.id=40f51bfd8db04611b068f7edd7002004, user.id=e22bb34e90cd4ea498ea37049b75163d) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:15.835882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27da007-5deb-4a41-a190-6a41b7ce8ef7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:15.837531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27da007-5deb-4a41-a190-6a41b7ce8ef7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:15.843936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 101/208] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:15 2022] GET /identity/v3/role_assignments?user.id=e22bb34e90cd4ea498ea37049b75163d&scope.project.id=40f51bfd8db04611b068f7edd7002004&role.id=368ae945e96746faa1dc2ecbcefcf0f1 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:17.053493 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a5b7f-6d7f-4ae5-896e-170dba2af025 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:17.054324 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a5b7f-6d7f-4ae5-896e-170dba2af025 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:17.054853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a5b7f-6d7f-4ae5-896e-170dba2af025 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:17.056708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 108/209] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:17.063603 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa29d61-eb89-488a-8bd5-705cdfd34ce7 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:17.064410 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa29d61-eb89-488a-8bd5-705cdfd34ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:17.064918 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa29d61-eb89-488a-8bd5-705cdfd34ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:17.082642 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-dfa29d61-eb89-488a-8bd5-705cdfd34ce7 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:17.084131 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dfa29d61-eb89-488a-8bd5-705cdfd34ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:17.146791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:17.154924 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 102/210] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:17.179361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a5b7f-6d7f-4ae5-896e-170dba2af025 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:17.206938 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66423f18-2d73-47fe-b350-632f77258dd8 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:17.207238 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66423f18-2d73-47fe-b350-632f77258dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:17.208027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66423f18-2d73-47fe-b350-632f77258dd8 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:17.210183 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 103/211] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 03:58:17.215461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087fc0d-fb8a-4278-8fbb-34015d7f3f3c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:17.215771 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087fc0d-fb8a-4278-8fbb-34015d7f3f3c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:17.216041 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087fc0d-fb8a-4278-8fbb-34015d7f3f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:17.234774 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3087fc0d-fb8a-4278-8fbb-34015d7f3f3c None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:17.235984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3087fc0d-fb8a-4278-8fbb-34015d7f3f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:17.294242 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a5b7f-6d7f-4ae5-896e-170dba2af025 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:17.309215 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 104/212] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:58:17.330840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9258e-cf29-4370-8532-f62c0c35ec54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:17.332003 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9258e-cf29-4370-8532-f62c0c35ec54 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:17.333105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9258e-cf29-4370-8532-f62c0c35ec54 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:17.333593 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9258e-cf29-4370-8532-f62c0c35ec54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:17.334737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087fc0d-fb8a-4278-8fbb-34015d7f3f3c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:17.345206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca9258e-cf29-4370-8532-f62c0c35ec54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:17.354364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-9ca9258e-cf29-4370-8532-f62c0c35ec54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:17.354364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:17.354364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:17.354364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:17.354364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:17.354364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:17.354364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:17.354364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:17.354364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:17.354364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:17.354364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:17.354364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:17.354364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:17.354364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:17.354364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:17.354364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:17.355299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:17.355299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:17.355299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:17.355299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:17.355299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:17.355299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:17.355299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:17.355299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:17.355299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:17.355299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:17.355299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:17.355299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:17.355299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:17.355299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:17.355299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:17.355299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:17.356081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:17.356081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:17.356081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:17.356081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:17.356081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:17.356081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:17.356081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:17.356081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:17.356081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:17.356081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:17.356081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:17.356081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:17.358878 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 109/213] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:17.365142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9258e-cf29-4370-8532-f62c0c35ec54 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:17.387267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:17.445046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3087fc0d-fb8a-4278-8fbb-34015d7f3f3c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:17.473494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:17.474201 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9258e-cf29-4370-8532-f62c0c35ec54 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:17.483111 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf27a2-9420-41f7-942e-d1ecb8501dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:17.484035 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf27a2-9420-41f7-942e-d1ecb8501dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:17.484329 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf27a2-9420-41f7-942e-d1ecb8501dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:17.484659 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf27a2-9420-41f7-942e-d1ecb8501dd6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:17.485869 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf27a2-9420-41f7-942e-d1ecb8501dd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:17.487515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bf27a2-9420-41f7-942e-d1ecb8501dd6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:17.489645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 105/214] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:17.495618 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf27a2-9420-41f7-942e-d1ecb8501dd6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:17.513939 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfd194-6caa-451e-b6ab-8822a0602a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:17.514989 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfd194-6caa-451e-b6ab-8822a0602a93 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:17.515518 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfd194-6caa-451e-b6ab-8822a0602a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:17.515998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfd194-6caa-451e-b6ab-8822a0602a93 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:17.516951 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfd194-6caa-451e-b6ab-8822a0602a93 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:17.517461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfd194-6caa-451e-b6ab-8822a0602a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:17.519093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfd194-6caa-451e-b6ab-8822a0602a93 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:17.528122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 110/215] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:17.533738 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfd194-6caa-451e-b6ab-8822a0602a93 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:17.610831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf27a2-9420-41f7-942e-d1ecb8501dd6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:17.622718 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:17.642980 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfd194-6caa-451e-b6ab-8822a0602a93 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:17.648450 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c80f0d-3a0e-40ac-bca8-c5e311f1200e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:17.649232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c80f0d-3a0e-40ac-bca8-c5e311f1200e None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:17.649519 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c80f0d-3a0e-40ac-bca8-c5e311f1200e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:17.649853 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c80f0d-3a0e-40ac-bca8-c5e311f1200e None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:17.650656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c80f0d-3a0e-40ac-bca8-c5e311f1200e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:17.652116 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c80f0d-3a0e-40ac-bca8-c5e311f1200e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:17.682958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bc1d2-0dba-4701-92d4-6c90a0ace90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:17.683965 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bc1d2-0dba-4701-92d4-6c90a0ace90c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:17.684329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bc1d2-0dba-4701-92d4-6c90a0ace90c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:17.684703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bc1d2-0dba-4701-92d4-6c90a0ace90c None admin] PATH_INFO: `/v3/users/0f4b38c1503d472195344d43a909a5df` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:17.698434 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632bc1d2-0dba-4701-92d4-6c90a0ace90c None admin] RBAC: Authorizing `identity:get_user(user_id=0f4b38c1503d472195344d43a909a5df)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:17.699917 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632bc1d2-0dba-4701-92d4-6c90a0ace90c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:17.701630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 111/216] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:17 2022] GET /identity/v3/users/0f4b38c1503d472195344d43a909a5df => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:17.703809 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb37744-50d1-49ed-862f-b0e7efc4fc9c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:17.704130 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb37744-50d1-49ed-862f-b0e7efc4fc9c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:17.704416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb37744-50d1-49ed-862f-b0e7efc4fc9c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:17.705894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 112/217] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:17.710536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb37744-50d1-49ed-862f-b0e7efc4fc9c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:17.743318 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 106/218] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jun 24 03:58:17 2022] POST /identity/v3/users => generated 297 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:58:17.745261 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d195428-50c0-4bc3-87b5-f7c233f6f81b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:17.745877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d195428-50c0-4bc3-87b5-f7c233f6f81b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:17.746159 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d195428-50c0-4bc3-87b5-f7c233f6f81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:17.765817 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4d195428-50c0-4bc3-87b5-f7c233f6f81b None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:17.767100 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4d195428-50c0-4bc3-87b5-f7c233f6f81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:17.820225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:17.827940 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb37744-50d1-49ed-862f-b0e7efc4fc9c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:17.846479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 107/219] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:17.848751 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47def6-3497-4518-85e0-7cebe5d588c0 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:17.849046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47def6-3497-4518-85e0-7cebe5d588c0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:17.849326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47def6-3497-4518-85e0-7cebe5d588c0 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:17.851020 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 108/220] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:17.859583 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b978279-49ab-4c5f-af1e-9fe16cac985f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:17.860073 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b978279-49ab-4c5f-af1e-9fe16cac985f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:17.860479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b978279-49ab-4c5f-af1e-9fe16cac985f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:17.880162 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403803a9-cd51-4cd7-afce-35c89a00a085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:17.881102 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403803a9-cd51-4cd7-afce-35c89a00a085 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:17.881567 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403803a9-cd51-4cd7-afce-35c89a00a085 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:17.881954 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8b978279-49ab-4c5f-af1e-9fe16cac985f None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:17.882128 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403803a9-cd51-4cd7-afce-35c89a00a085 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:17.883286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8b978279-49ab-4c5f-af1e-9fe16cac985f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:17.893215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:17.961957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 109/221] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:17.962786 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b978279-49ab-4c5f-af1e-9fe16cac985f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:17.964711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403803a9-cd51-4cd7-afce-35c89a00a085 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:17.966194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403803a9-cd51-4cd7-afce-35c89a00a085 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:18.095898 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b978279-49ab-4c5f-af1e-9fe16cac985f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:18.101426 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 113/222] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:18.102098 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403803a9-cd51-4cd7-afce-35c89a00a085 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:18.107531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:18.141332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a155ff-6eab-4a9a-96f8-eeaf54a9cb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:18.142629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a155ff-6eab-4a9a-96f8-eeaf54a9cb00 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:18.143200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a155ff-6eab-4a9a-96f8-eeaf54a9cb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:18.143678 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a155ff-6eab-4a9a-96f8-eeaf54a9cb00 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:18.149352 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:18.156820 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a155ff-6eab-4a9a-96f8-eeaf54a9cb00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:18.166066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-11a155ff-6eab-4a9a-96f8-eeaf54a9cb00 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:18.166066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:18.166066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:18.166066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:18.166066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:18.166066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:18.166066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:18.166066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:18.166066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:18.166066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:18.166066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:18.166066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:18.166066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:18.166066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:18.166066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:18.166066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:18.167218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:18.167218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:18.167218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:18.167218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:18.167218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:18.167218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:18.167218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:18.167218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:18.167218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:18.167218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:18.167218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:18.167218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:18.167218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:18.167218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:18.167218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:18.167218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:18.168032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:18.168032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:18.168032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:18.168032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:18.168032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:18.168032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:18.168032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:18.168032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:18.168032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:18.168032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:18.168032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:18.168032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:18.170509 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 110/223] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:58:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:18.171874 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a155ff-6eab-4a9a-96f8-eeaf54a9cb00 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:18.212015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403803a9-cd51-4cd7-afce-35c89a00a085 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:18.249673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17321fe-ad55-438b-9e7c-be8cbbd5241b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:18.250745 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17321fe-ad55-438b-9e7c-be8cbbd5241b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:18.250875 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17321fe-ad55-438b-9e7c-be8cbbd5241b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:18.251101 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17321fe-ad55-438b-9e7c-be8cbbd5241b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:18.262354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17321fe-ad55-438b-9e7c-be8cbbd5241b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:18.275438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-b17321fe-ad55-438b-9e7c-be8cbbd5241b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:58:18.275438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:18.275438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:18.275438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:18.275438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:18.275438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:18.275438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:18.275438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:18.275438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:18.275438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:18.275438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:18.275438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:18.275438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:18.275438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:18.275438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:18.275438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:18.276367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:18.276367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:18.276367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:18.276367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:18.276367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:18.276367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:18.276367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:18.276367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:18.276367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:18.276367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:18.276367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:18.276367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:18.276367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:18.276367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:18.276367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:18.276367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:18.277588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:18.277588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:18.277588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:18.277588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:18.277588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:18.277588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:18.277588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:18.277588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:18.277588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:18.277588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:18.277588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:58:18.277588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:18.277588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 114/224] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:58:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:18.277588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17321fe-ad55-438b-9e7c-be8cbbd5241b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:18.301881 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a155ff-6eab-4a9a-96f8-eeaf54a9cb00 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:18.340311 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a8524-027b-4128-8cd7-0961b0df0265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:18.341207 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a8524-027b-4128-8cd7-0961b0df0265 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:18.341494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a8524-027b-4128-8cd7-0961b0df0265 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:18.341778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a8524-027b-4128-8cd7-0961b0df0265 None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:18.343065 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a8524-027b-4128-8cd7-0961b0df0265 None admin] RBAC: Authorizing `identity:get_project(project_id=e87c7eebde3941f1aa5f40ce794e8687)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:18.344635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a8524-027b-4128-8cd7-0961b0df0265 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:18.351330 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 111/225] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:18 2022] GET /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:18.351330 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a8524-027b-4128-8cd7-0961b0df0265 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:18.397153 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:18.421642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:18.427078 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17321fe-ad55-438b-9e7c-be8cbbd5241b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:18.466226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a8524-027b-4128-8cd7-0961b0df0265 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:18.474869 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08d278-a73f-49ed-827c-1b8fc6bfd86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:18.475564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08d278-a73f-49ed-827c-1b8fc6bfd86f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:18.475859 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08d278-a73f-49ed-827c-1b8fc6bfd86f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:18.476800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08d278-a73f-49ed-827c-1b8fc6bfd86f None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:18.477583 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08d278-a73f-49ed-827c-1b8fc6bfd86f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:18.477908 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08d278-a73f-49ed-827c-1b8fc6bfd86f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:18.479837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08d278-a73f-49ed-827c-1b8fc6bfd86f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:18.490801 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 115/226] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:58:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:18.491668 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08d278-a73f-49ed-827c-1b8fc6bfd86f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:18.522309 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7b596-1a25-49cb-a00e-0c642e8ffd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:18.523334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7b596-1a25-49cb-a00e-0c642e8ffd0e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:18.524055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7b596-1a25-49cb-a00e-0c642e8ffd0e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:18.524588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7b596-1a25-49cb-a00e-0c642e8ffd0e None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:18.525616 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7b596-1a25-49cb-a00e-0c642e8ffd0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:18.526218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7b596-1a25-49cb-a00e-0c642e8ffd0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:18.528359 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7b596-1a25-49cb-a00e-0c642e8ffd0e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:18.543009 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 112/227] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:18.543984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7b596-1a25-49cb-a00e-0c642e8ffd0e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:18.621918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08d278-a73f-49ed-827c-1b8fc6bfd86f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:18.668031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659208fe-2c80-4e09-bbb5-e9cbf8ef9e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:18.668991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659208fe-2c80-4e09-bbb5-e9cbf8ef9e83 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:18.669415 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659208fe-2c80-4e09-bbb5-e9cbf8ef9e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:18.669840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659208fe-2c80-4e09-bbb5-e9cbf8ef9e83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:18.670936 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659208fe-2c80-4e09-bbb5-e9cbf8ef9e83 None admin] RBAC: Adding query filter params (role.id=d7aadc20cd4c4838ae5d7d8e8826abae, scope.project.id=e87c7eebde3941f1aa5f40ce794e8687, user.id=0f4b38c1503d472195344d43a909a5df) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:18.671404 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659208fe-2c80-4e09-bbb5-e9cbf8ef9e83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:18.673104 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659208fe-2c80-4e09-bbb5-e9cbf8ef9e83 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:18.680408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 116/228] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:18 2022] GET /identity/v3/role_assignments?user.id=0f4b38c1503d472195344d43a909a5df&scope.project.id=e87c7eebde3941f1aa5f40ce794e8687&role.id=d7aadc20cd4c4838ae5d7d8e8826abae => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:18.681697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659208fe-2c80-4e09-bbb5-e9cbf8ef9e83 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:18.699258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:18.710413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7b596-1a25-49cb-a00e-0c642e8ffd0e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:18.714728 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:18.779700 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6a6df-d003-4689-881c-4b753e569a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:18.781185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6a6df-d003-4689-881c-4b753e569a1c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:18.781538 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6a6df-d003-4689-881c-4b753e569a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:18.781826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b6a6df-d003-4689-881c-4b753e569a1c None admin] PATH_INFO: `/v3/users/381eba1a68b54e2eabaf4f64a95e41cd` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:18.786498 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b6a6df-d003-4689-881c-4b753e569a1c None admin] RBAC: Authorizing `identity:get_user(user_id=381eba1a68b54e2eabaf4f64a95e41cd)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:18.788107 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b6a6df-d003-4689-881c-4b753e569a1c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:18.790438 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 113/229] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:18 2022] GET /identity/v3/users/381eba1a68b54e2eabaf4f64a95e41cd => generated 283 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:18.799728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6a6df-d003-4689-881c-4b753e569a1c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:18.818413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659208fe-2c80-4e09-bbb5-e9cbf8ef9e83 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:18.859927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d7c83-4ed6-431c-b684-b6bbf6ed248d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:18.861333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d7c83-4ed6-431c-b684-b6bbf6ed248d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:18.861621 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d7c83-4ed6-431c-b684-b6bbf6ed248d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:18.861907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d7c83-4ed6-431c-b684-b6bbf6ed248d None admin] PATH_INFO: `/v3/users/381eba1a68b54e2eabaf4f64a95e41cd` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:18.863460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d7c83-4ed6-431c-b684-b6bbf6ed248d None admin] RBAC: Authorizing `identity:get_user(user_id=381eba1a68b54e2eabaf4f64a95e41cd)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:18.865315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d7c83-4ed6-431c-b684-b6bbf6ed248d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:18.870419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 117/230] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:18 2022] GET /identity/v3/users/381eba1a68b54e2eabaf4f64a95e41cd => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:18.873913 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d7c83-4ed6-431c-b684-b6bbf6ed248d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:18.968702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b6a6df-d003-4689-881c-4b753e569a1c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:19.021383 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6840470-e90c-4063-a8ee-ec0bc9ff204f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:19.022525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6840470-e90c-4063-a8ee-ec0bc9ff204f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:19.022970 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6840470-e90c-4063-a8ee-ec0bc9ff204f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:19.023660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6840470-e90c-4063-a8ee-ec0bc9ff204f None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:19.025194 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6840470-e90c-4063-a8ee-ec0bc9ff204f None admin] RBAC: Authorizing `identity:get_project(project_id=e87c7eebde3941f1aa5f40ce794e8687)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:19.027652 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6840470-e90c-4063-a8ee-ec0bc9ff204f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:19.029888 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 114/231] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:18 2022] GET /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:19.037104 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6840470-e90c-4063-a8ee-ec0bc9ff204f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:19.084623 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:19.115931 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:19.137268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d7c83-4ed6-431c-b684-b6bbf6ed248d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:19.199093 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6840470-e90c-4063-a8ee-ec0bc9ff204f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:19.208081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6915bd-22bd-4954-8e4c-eb98d597f992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:19.215221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6915bd-22bd-4954-8e4c-eb98d597f992 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:19.215706 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6915bd-22bd-4954-8e4c-eb98d597f992 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:19.216149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6915bd-22bd-4954-8e4c-eb98d597f992 None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:19.217956 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6915bd-22bd-4954-8e4c-eb98d597f992 None admin] RBAC: Authorizing `identity:get_project(project_id=e87c7eebde3941f1aa5f40ce794e8687)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:19.219733 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6915bd-22bd-4954-8e4c-eb98d597f992 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:19.221665 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 118/232] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:18 2022] GET /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687 => generated 302 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:19.228875 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6915bd-22bd-4954-8e4c-eb98d597f992 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:19.296507 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70493210-2121-4ae5-a13f-c1f9fd681cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:19.297679 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70493210-2121-4ae5-a13f-c1f9fd681cad None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:19.297679 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70493210-2121-4ae5-a13f-c1f9fd681cad None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:19.297971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70493210-2121-4ae5-a13f-c1f9fd681cad None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687/users/381eba1a68b54e2eabaf4f64a95e41cd/roles/a048041c27f44edf98a110d23138a327` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:19.305008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70493210-2121-4ae5-a13f-c1f9fd681cad None admin] RBAC: Authorizing `identity:create_grant(project_id=e87c7eebde3941f1aa5f40ce794e8687, user_id=381eba1a68b54e2eabaf4f64a95e41cd, role_id=a048041c27f44edf98a110d23138a327)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:19.306761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70493210-2121-4ae5-a13f-c1f9fd681cad None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:19.377038 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6915bd-22bd-4954-8e4c-eb98d597f992 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:19.426451 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 115/233] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 03:58:19 2022] PUT /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687/users/381eba1a68b54e2eabaf4f64a95e41cd/roles/a048041c27f44edf98a110d23138a327 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:58:19.427885 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ac8ff-14c9-4d50-b2d1-5e61114d7b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:19.429144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ac8ff-14c9-4d50-b2d1-5e61114d7b5d None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:19.429886 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ac8ff-14c9-4d50-b2d1-5e61114d7b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:19.430746 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ac8ff-14c9-4d50-b2d1-5e61114d7b5d None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687/users/381eba1a68b54e2eabaf4f64a95e41cd/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:19.433253 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ac8ff-14c9-4d50-b2d1-5e61114d7b5d None admin] RBAC: Authorizing `identity:create_grant(project_id=e87c7eebde3941f1aa5f40ce794e8687, user_id=381eba1a68b54e2eabaf4f64a95e41cd, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:19.435147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ac8ff-14c9-4d50-b2d1-5e61114d7b5d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:19.524780 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 119/234] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 03:58:19 2022] PUT /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687/users/381eba1a68b54e2eabaf4f64a95e41cd/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:58:20.184400 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c263a6-6dbf-4538-aa30-c74832e394ce None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:20.185086 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c263a6-6dbf-4538-aa30-c74832e394ce None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:20.185586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c263a6-6dbf-4538-aa30-c74832e394ce None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:20.187540 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 116/235] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:20.192880 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb442a-b52a-4c47-8d09-b5927412bfa5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:20.193364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb442a-b52a-4c47-8d09-b5927412bfa5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:20.193441 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb442a-b52a-4c47-8d09-b5927412bfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:20.217277 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-56fb442a-b52a-4c47-8d09-b5927412bfa5 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:20.218553 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-56fb442a-b52a-4c47-8d09-b5927412bfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:20.239423 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:20.320412 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 120/236] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:20.356072 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c263a6-6dbf-4538-aa30-c74832e394ce None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:20.406855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad2e86-b198-4c65-9a69-9cac7c65512e None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:20.407169 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad2e86-b198-4c65-9a69-9cac7c65512e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:20.407486 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad2e86-b198-4c65-9a69-9cac7c65512e None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:20.408874 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 121/237] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 03:58:20.414885 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42e795-6df9-4087-a500-81078f847b29 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:20.415173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42e795-6df9-4087-a500-81078f847b29 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:20.415448 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42e795-6df9-4087-a500-81078f847b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:20.432545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ed42e795-6df9-4087-a500-81078f847b29 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:20.433726 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ed42e795-6df9-4087-a500-81078f847b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:20.525436 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 122/238] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:58:20.542742 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:20.547911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c263a6-6dbf-4538-aa30-c74832e394ce None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:20.600294 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f0ccb-225c-4e87-ba27-5b58ecc0ed2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:20.601019 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f0ccb-225c-4e87-ba27-5b58ecc0ed2c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:20.601300 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f0ccb-225c-4e87-ba27-5b58ecc0ed2c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:20.601570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f0ccb-225c-4e87-ba27-5b58ecc0ed2c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:20.609018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42e795-6df9-4087-a500-81078f847b29 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:20.625543 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f0ccb-225c-4e87-ba27-5b58ecc0ed2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:20.633837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-467f0ccb-225c-4e87-ba27-5b58ecc0ed2c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 03:58:20.633837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:20.633837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:20.633837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:20.633837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:20.633837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:20.633837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:20.633837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:20.633837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:20.633837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:20.633837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:20.633837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:20.633837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:20.633837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:20.633837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:20.633837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:20.634754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:20.634754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:20.634754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:20.634754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:20.634754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:20.634754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:20.634754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:20.634754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:20.634754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:20.634754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:20.634754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:20.634754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:20.634754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:20.634754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:20.634754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:20.634754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:20.635552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:20.635552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:20.635552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:20.635552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:20.635552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:20.635552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:20.635552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:20.635552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:20.635552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:20.635552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:20.635552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 03:58:20.635552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:20.647298 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 117/239] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:20.650119 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f0ccb-225c-4e87-ba27-5b58ecc0ed2c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:20.754868 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:20.757969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f0ccb-225c-4e87-ba27-5b58ecc0ed2c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:20.778709 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42e795-6df9-4087-a500-81078f847b29 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:20.798414 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9180d-e53e-42da-a2b9-0f4c125e8c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:20.799088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9180d-e53e-42da-a2b9-0f4c125e8c5c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:20.799371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9180d-e53e-42da-a2b9-0f4c125e8c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:20.799646 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9180d-e53e-42da-a2b9-0f4c125e8c5c None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:20.800415 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9180d-e53e-42da-a2b9-0f4c125e8c5c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:20.800728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9180d-e53e-42da-a2b9-0f4c125e8c5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:20.802551 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9180d-e53e-42da-a2b9-0f4c125e8c5c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:20.825647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 118/240] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:20.839105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9180d-e53e-42da-a2b9-0f4c125e8c5c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:20.852071 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe61db-bb47-45fe-9d71-75a9d081b9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:20.852828 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe61db-bb47-45fe-9d71-75a9d081b9fc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:20.853117 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe61db-bb47-45fe-9d71-75a9d081b9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:20.853398 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe61db-bb47-45fe-9d71-75a9d081b9fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:20.854639 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe61db-bb47-45fe-9d71-75a9d081b9fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:20.856107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe61db-bb47-45fe-9d71-75a9d081b9fc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:20.857973 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 123/241] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:20.861022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:20.863349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe61db-bb47-45fe-9d71-75a9d081b9fc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:21.001544 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe61db-bb47-45fe-9d71-75a9d081b9fc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:21.041017 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2baab-c9c6-4777-aa5d-9aba6cc7d89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:21.041017 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2baab-c9c6-4777-aa5d-9aba6cc7d89c None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:21.041017 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2baab-c9c6-4777-aa5d-9aba6cc7d89c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:21.041370 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2baab-c9c6-4777-aa5d-9aba6cc7d89c None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:21.042099 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2baab-c9c6-4777-aa5d-9aba6cc7d89c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:21.044041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2baab-c9c6-4777-aa5d-9aba6cc7d89c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:21.066332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9180d-e53e-42da-a2b9-0f4c125e8c5c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:21.107812 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 124/242] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jun 24 03:58:20 2022] POST /identity/v3/users => generated 293 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:58:21.109017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6ba7e-d797-431d-9913-e46460a19f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:21.110035 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6ba7e-d797-431d-9913-e46460a19f4a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:21.110324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6ba7e-d797-431d-9913-e46460a19f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:21.110923 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6ba7e-d797-431d-9913-e46460a19f4a None admin] PATH_INFO: `/v3/users/c7419dda6a87453486513e85344eac1f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:21.128659 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6ba7e-d797-431d-9913-e46460a19f4a None admin] RBAC: Authorizing `identity:get_user(user_id=c7419dda6a87453486513e85344eac1f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:21.131020 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6ba7e-d797-431d-9913-e46460a19f4a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:21.133247 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 119/243] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:20 2022] GET /identity/v3/users/c7419dda6a87453486513e85344eac1f => generated 297 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:21.138848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2baab-c9c6-4777-aa5d-9aba6cc7d89c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:21.143045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:21.280417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2baab-c9c6-4777-aa5d-9aba6cc7d89c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:21.316043 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6c015-9435-4156-8de6-78eb7db8c292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:21.316744 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6c015-9435-4156-8de6-78eb7db8c292 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:21.317097 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6c015-9435-4156-8de6-78eb7db8c292 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:21.317484 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6c015-9435-4156-8de6-78eb7db8c292 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:21.391065 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6c015-9435-4156-8de6-78eb7db8c292 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:21.392554 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb6c015-9435-4156-8de6-78eb7db8c292 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:21.395490 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:21.467984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997c3ac-5692-40bd-ab8d-be63e43fc57e None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:21.468532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997c3ac-5692-40bd-ab8d-be63e43fc57e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:21.468532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f997c3ac-5692-40bd-ab8d-be63e43fc57e None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:21.470226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 120/244] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:21.477799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c9a77-8a25-4246-ba23-456a909426e4 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:21.478075 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c9a77-8a25-4246-ba23-456a909426e4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:21.478357 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c9a77-8a25-4246-ba23-456a909426e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:21.518665 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-762c9a77-8a25-4246-ba23-456a909426e4 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:21.518665 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-762c9a77-8a25-4246-ba23-456a909426e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:21.540663 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 125/245] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:21.547785 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6c015-9435-4156-8de6-78eb7db8c292 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:21.674501 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 121/246] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:21.712257 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6c015-9435-4156-8de6-78eb7db8c292 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:21.715224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:21.738430 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c9a77-8a25-4246-ba23-456a909426e4 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:21.751711 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:21.763226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb7698-0262-47a5-a930-87b69dcb206a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:21.764214 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb7698-0262-47a5-a930-87b69dcb206a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:21.764503 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb7698-0262-47a5-a930-87b69dcb206a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:21.764781 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb7698-0262-47a5-a930-87b69dcb206a None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:21.770574 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb7698-0262-47a5-a930-87b69dcb206a None admin] RBAC: Authorizing `identity:get_project(project_id=e87c7eebde3941f1aa5f40ce794e8687)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:21.773308 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb7698-0262-47a5-a930-87b69dcb206a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:21.776558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 126/247] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:21 2022] GET /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:21.797474 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb7698-0262-47a5-a930-87b69dcb206a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:21.954697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb7698-0262-47a5-a930-87b69dcb206a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:21.966306 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c9a77-8a25-4246-ba23-456a909426e4 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:22.007312 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846110c8-891b-44a4-93ee-968837fc81fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:22.007998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846110c8-891b-44a4-93ee-968837fc81fa None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:22.008282 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846110c8-891b-44a4-93ee-968837fc81fa None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:22.008561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846110c8-891b-44a4-93ee-968837fc81fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:22.010072 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846110c8-891b-44a4-93ee-968837fc81fa None admin] RBAC: Adding query filter params (role.id=368ae945e96746faa1dc2ecbcefcf0f1, scope.project.id=e87c7eebde3941f1aa5f40ce794e8687, user.id=c7419dda6a87453486513e85344eac1f) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:22.010669 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846110c8-891b-44a4-93ee-968837fc81fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:22.018176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846110c8-891b-44a4-93ee-968837fc81fa None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:22.027065 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 127/248] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:21 2022] GET /identity/v3/role_assignments?user.id=c7419dda6a87453486513e85344eac1f&scope.project.id=e87c7eebde3941f1aa5f40ce794e8687&role.id=368ae945e96746faa1dc2ecbcefcf0f1 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:22.027224 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ecf4c-d30a-4040-acb8-7d52a37a4c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:22.030432 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ecf4c-d30a-4040-acb8-7d52a37a4c92 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:22.030432 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ecf4c-d30a-4040-acb8-7d52a37a4c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:22.030432 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ecf4c-d30a-4040-acb8-7d52a37a4c92 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:22.050452 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ecf4c-d30a-4040-acb8-7d52a37a4c92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:22.059137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-024ecf4c-d30a-4040-acb8-7d52a37a4c92 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:22.059137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:22.059137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:22.059137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:22.059137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:22.059137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:22.059137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:22.059137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:22.059137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:22.059137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:22.059137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:22.059137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:22.059137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:22.059137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:22.059137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:22.059137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:22.060113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:22.060113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:22.060113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:22.060113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:22.060113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:22.060113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:22.060113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:22.060113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:22.060113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:22.060113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:22.060113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:22.060113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:22.060113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:22.060113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:22.060113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:22.060113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:22.060909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:22.060909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:22.060909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:22.060909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:22.060909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:22.060909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:22.060909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:22.060909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:22.060909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:22.060909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:22.060909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:22.060909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:22.062874 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 122/249] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:58:22.067736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846110c8-891b-44a4-93ee-968837fc81fa None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:22.102571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e021c-1135-4de1-980b-a6bb4e478396 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:22.102907 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e021c-1135-4de1-980b-a6bb4e478396 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:22.103152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e021c-1135-4de1-980b-a6bb4e478396 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:22.109495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:22.110879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 123/250] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:22.119316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5559fb4-11cb-4381-bc9c-ad983e5af937 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:22.119731 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5559fb4-11cb-4381-bc9c-ad983e5af937 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:22.120056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5559fb4-11cb-4381-bc9c-ad983e5af937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:22.159636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f5559fb4-11cb-4381-bc9c-ad983e5af937 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:22.161371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f5559fb4-11cb-4381-bc9c-ad983e5af937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:22.187075 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846110c8-891b-44a4-93ee-968837fc81fa None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:22.226316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554c78c-90c9-4f31-9aa4-36b528884c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:22.226786 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554c78c-90c9-4f31-9aa4-36b528884c46 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:22.227079 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554c78c-90c9-4f31-9aa4-36b528884c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:22.227398 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554c78c-90c9-4f31-9aa4-36b528884c46 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:22.228140 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554c78c-90c9-4f31-9aa4-36b528884c46 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:22.228463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554c78c-90c9-4f31-9aa4-36b528884c46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:22.229909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554c78c-90c9-4f31-9aa4-36b528884c46 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:22.244452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 128/251] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:22.246941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554c78c-90c9-4f31-9aa4-36b528884c46 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:22.335641 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:22.359391 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 124/252] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:58:22.366630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:22.371967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554c78c-90c9-4f31-9aa4-36b528884c46 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:22.402072 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5559fb4-11cb-4381-bc9c-ad983e5af937 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:22.442449 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7c972-adc8-4980-bff3-e335a0382b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:22.442449 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7c972-adc8-4980-bff3-e335a0382b23 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:22.442449 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7c972-adc8-4980-bff3-e335a0382b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:22.442449 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7c972-adc8-4980-bff3-e335a0382b23 None admin] PATH_INFO: `/v3/users/0f4b38c1503d472195344d43a909a5df` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:22.442814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7c972-adc8-4980-bff3-e335a0382b23 None admin] RBAC: Authorizing `identity:get_user(user_id=0f4b38c1503d472195344d43a909a5df)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:22.442814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7c972-adc8-4980-bff3-e335a0382b23 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:22.444163 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 129/253] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:22 2022] GET /identity/v3/users/0f4b38c1503d472195344d43a909a5df => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:22.446025 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3116c-1b5c-4be4-853d-551b7e89fff3 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:22.446448 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3116c-1b5c-4be4-853d-551b7e89fff3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:22.446822 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3116c-1b5c-4be4-853d-551b7e89fff3 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:22.448547 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 130/254] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 03:58:22.449266 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3116c-1b5c-4be4-853d-551b7e89fff3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:22.541712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5559fb4-11cb-4381-bc9c-ad983e5af937 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:22.595765 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82eea4-b783-4703-8498-c613b3dc8292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:22.599840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82eea4-b783-4703-8498-c613b3dc8292 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:22.600385 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82eea4-b783-4703-8498-c613b3dc8292 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:22.600867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82eea4-b783-4703-8498-c613b3dc8292 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:22.620197 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82eea4-b783-4703-8498-c613b3dc8292 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:22.629743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-ab82eea4-b783-4703-8498-c613b3dc8292 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:22.629743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:22.629743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:22.629743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:22.629743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:22.629743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:22.629743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:22.629743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:22.629743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:22.629743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:22.629743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:22.629743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:22.629743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:22.629743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:22.629743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:22.629743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:22.630696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:22.630696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:22.630696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:22.630696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:22.630696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:22.630696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:22.630696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:22.630696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:22.630696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:22.630696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:22.630696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:22.630696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:22.630696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:22.630696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:22.630696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:22.630696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:22.631494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:22.631494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:22.631494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:22.631494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:22.631494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:22.631494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:22.631494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:22.631494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:22.631494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:22.631494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:22.631494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:22.631494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:22.634328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 125/255] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:58:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:58:22.636108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94bba0-d7a5-493a-a807-5d5a9d8d5d72 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:22.636700 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94bba0-d7a5-493a-a807-5d5a9d8d5d72 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:22.637176 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94bba0-d7a5-493a-a807-5d5a9d8d5d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:22.648096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3116c-1b5c-4be4-853d-551b7e89fff3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:22.655924 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3b94bba0-d7a5-493a-a807-5d5a9d8d5d72 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:22.657372 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3b94bba0-d7a5-493a-a807-5d5a9d8d5d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:22.713213 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a943b-cae8-41e4-80e9-425f023fc31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:22.714660 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a943b-cae8-41e4-80e9-425f023fc31f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:22.716118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a943b-cae8-41e4-80e9-425f023fc31f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:22.716813 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a943b-cae8-41e4-80e9-425f023fc31f None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:22.719207 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a943b-cae8-41e4-80e9-425f023fc31f None admin] RBAC: Authorizing `identity:get_project(project_id=e87c7eebde3941f1aa5f40ce794e8687)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:22.721500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a943b-cae8-41e4-80e9-425f023fc31f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:22.727911 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 131/256] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:22 2022] GET /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:22.728950 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a943b-cae8-41e4-80e9-425f023fc31f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:22.761241 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:22.793668 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 126/257] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:22.796950 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94bba0-d7a5-493a-a807-5d5a9d8d5d72 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:22.797952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:22.929117 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a943b-cae8-41e4-80e9-425f023fc31f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:22.943880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94bba0-d7a5-493a-a807-5d5a9d8d5d72 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:22.981758 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfb2a3-efad-41b5-abae-05b0b1a4dd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:22.983007 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfb2a3-efad-41b5-abae-05b0b1a4dd35 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:22.983494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfb2a3-efad-41b5-abae-05b0b1a4dd35 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:22.983989 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfb2a3-efad-41b5-abae-05b0b1a4dd35 None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687/users/0f4b38c1503d472195344d43a909a5df/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:22.986502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cfb2a3-efad-41b5-abae-05b0b1a4dd35 None admin] RBAC: Authorizing `identity:create_grant(project_id=e87c7eebde3941f1aa5f40ce794e8687, user_id=0f4b38c1503d472195344d43a909a5df, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:22.988242 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cfb2a3-efad-41b5-abae-05b0b1a4dd35 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:22.997340 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeae874-4948-4a6b-83cd-6edb1f0fe03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:22.998424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeae874-4948-4a6b-83cd-6edb1f0fe03f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:22.998920 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeae874-4948-4a6b-83cd-6edb1f0fe03f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:22.999423 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeae874-4948-4a6b-83cd-6edb1f0fe03f None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:23.000438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeae874-4948-4a6b-83cd-6edb1f0fe03f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:23.000969 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeae874-4948-4a6b-83cd-6edb1f0fe03f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:23.002798 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeae874-4948-4a6b-83cd-6edb1f0fe03f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:23.012542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 132/258] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:58:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:23.013918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeae874-4948-4a6b-83cd-6edb1f0fe03f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:23.062441 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 127/259] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 03:58:22 2022] PUT /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687/users/0f4b38c1503d472195344d43a909a5df/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:58:23.063736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfb2a3-efad-41b5-abae-05b0b1a4dd35 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:23.178690 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:23.229224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:23.244128 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeae874-4948-4a6b-83cd-6edb1f0fe03f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:23.291253 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cfb2a3-efad-41b5-abae-05b0b1a4dd35 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:23.309204 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8d9ef-5b64-4fd3-a3b5-7c21320ae0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:23.310403 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8d9ef-5b64-4fd3-a3b5-7c21320ae0db None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:23.312945 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8d9ef-5b64-4fd3-a3b5-7c21320ae0db None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:23.313467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8d9ef-5b64-4fd3-a3b5-7c21320ae0db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:23.340481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8d9ef-5b64-4fd3-a3b5-7c21320ae0db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:23.356258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-b8c8d9ef-5b64-4fd3-a3b5-7c21320ae0db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:58:23.356258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:23.356258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:23.356258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:23.356258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:23.356258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:23.356258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:23.356258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:23.356258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:23.356258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:23.356258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:23.356258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:23.356258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:23.356258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:23.356258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:23.356258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:23.357325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:23.357325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:23.357325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:23.357325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:23.357325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:23.357325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:23.357325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:23.357325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:23.357325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:23.357325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:23.357325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:23.357325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:23.357325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:23.357325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:23.357325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:23.357325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:23.358147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:23.358147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:23.358147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:23.358147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:23.358147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:23.358147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:23.358147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:23.358147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:23.358147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:23.358147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:23.358147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:58:23.358147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:23.363281 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 133/260] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:58:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:23.369859 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8d9ef-5b64-4fd3-a3b5-7c21320ae0db None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:23.373916 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb12a3-f721-46db-b073-d5a2915751b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:23.375452 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb12a3-f721-46db-b073-d5a2915751b7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:23.376166 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb12a3-f721-46db-b073-d5a2915751b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:23.377555 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb12a3-f721-46db-b073-d5a2915751b7 None admin] PATH_INFO: `/v3/users/381eba1a68b54e2eabaf4f64a95e41cd` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:23.387454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb12a3-f721-46db-b073-d5a2915751b7 None admin] RBAC: Authorizing `identity:get_user(user_id=381eba1a68b54e2eabaf4f64a95e41cd)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:23.388917 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb12a3-f721-46db-b073-d5a2915751b7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:23.390970 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 128/261] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:23 2022] GET /identity/v3/users/381eba1a68b54e2eabaf4f64a95e41cd => generated 283 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:23.397113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb12a3-f721-46db-b073-d5a2915751b7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:23.539449 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8d9ef-5b64-4fd3-a3b5-7c21320ae0db None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:23.604618 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb12a3-f721-46db-b073-d5a2915751b7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:23.610944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfafe2-a89a-4eb2-b14a-b06d510b5130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:23.611629 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfafe2-a89a-4eb2-b14a-b06d510b5130 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:23.611915 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfafe2-a89a-4eb2-b14a-b06d510b5130 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:23.614397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfafe2-a89a-4eb2-b14a-b06d510b5130 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:23.614397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfafe2-a89a-4eb2-b14a-b06d510b5130 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:23.614397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfafe2-a89a-4eb2-b14a-b06d510b5130 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:23.616031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfafe2-a89a-4eb2-b14a-b06d510b5130 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:23.637320 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:23.637917 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 134/262] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:23.642056 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfafe2-a89a-4eb2-b14a-b06d510b5130 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:23.672560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:23.673160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33141b-c259-40a1-acf6-0af346dcd8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:23.674064 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33141b-c259-40a1-acf6-0af346dcd8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:23.674355 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33141b-c259-40a1-acf6-0af346dcd8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:23.674665 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33141b-c259-40a1-acf6-0af346dcd8c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:23.784911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33141b-c259-40a1-acf6-0af346dcd8c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:23.788004 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33141b-c259-40a1-acf6-0af346dcd8c9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:23.822939 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfafe2-a89a-4eb2-b14a-b06d510b5130 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:23.909484 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f9567-e512-4be5-9977-d6e1ed7878ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:23.911743 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9567-e512-4be5-9977-d6e1ed7878ec None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:23.914947 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9567-e512-4be5-9977-d6e1ed7878ec None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:23.915436 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9567-e512-4be5-9977-d6e1ed7878ec None admin] PATH_INFO: `/v3/users/381eba1a68b54e2eabaf4f64a95e41cd` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:23.917100 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f9567-e512-4be5-9977-d6e1ed7878ec None admin] RBAC: Authorizing `identity:get_user(user_id=381eba1a68b54e2eabaf4f64a95e41cd)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:23.919768 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f9567-e512-4be5-9977-d6e1ed7878ec None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:23.921804 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 135/263] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:23 2022] GET /identity/v3/users/381eba1a68b54e2eabaf4f64a95e41cd => generated 283 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:23.929153 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f9567-e512-4be5-9977-d6e1ed7878ec None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:24.042109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 129/264] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:24.049455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33141b-c259-40a1-acf6-0af346dcd8c9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:24.082581 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:24.162138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:24.191352 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f9567-e512-4be5-9977-d6e1ed7878ec None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:24.276264 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58bec3-1d8b-4219-9ccf-20f2b25089c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:24.276931 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58bec3-1d8b-4219-9ccf-20f2b25089c1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:24.277213 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58bec3-1d8b-4219-9ccf-20f2b25089c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:24.277486 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58bec3-1d8b-4219-9ccf-20f2b25089c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:24.317240 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33141b-c259-40a1-acf6-0af346dcd8c9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:24.390135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef58bec3-1d8b-4219-9ccf-20f2b25089c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:24.391668 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef58bec3-1d8b-4219-9ccf-20f2b25089c1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:24.393980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36543fee-e2a3-4ef4-bf40-3a1a7be71f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:24.396127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36543fee-e2a3-4ef4-bf40-3a1a7be71f96 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:24.396413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36543fee-e2a3-4ef4-bf40-3a1a7be71f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:24.396691 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36543fee-e2a3-4ef4-bf40-3a1a7be71f96 None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:24.397929 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36543fee-e2a3-4ef4-bf40-3a1a7be71f96 None admin] RBAC: Authorizing `identity:get_project(project_id=e87c7eebde3941f1aa5f40ce794e8687)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:24.399431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36543fee-e2a3-4ef4-bf40-3a1a7be71f96 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:24.401581 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 130/265] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:24 2022] GET /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687 => generated 302 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:24.407182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e9274-55c1-4bd6-ae62-c1c28e623d4e None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:24.407522 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e9274-55c1-4bd6-ae62-c1c28e623d4e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:24.407806 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e9274-55c1-4bd6-ae62-c1c28e623d4e None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:24.409356 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 131/266] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:24.412378 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e9274-55c1-4bd6-ae62-c1c28e623d4e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:24.507149 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:24.511704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:24.532034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e9274-55c1-4bd6-ae62-c1c28e623d4e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:24.533943 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 136/267] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:24.536103 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c688e-ed6e-4d8c-8f8c-3bee2c4a9f39 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:24.536103 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c688e-ed6e-4d8c-8f8c-3bee2c4a9f39 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:24.536103 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c688e-ed6e-4d8c-8f8c-3bee2c4a9f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:24.554647 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-207c688e-ed6e-4d8c-8f8c-3bee2c4a9f39 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:24.555837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-207c688e-ed6e-4d8c-8f8c-3bee2c4a9f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:24.571179 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1abc2a-2eba-438a-bd6b-ff26a656e153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:24.572123 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1abc2a-2eba-438a-bd6b-ff26a656e153 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:24.572622 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1abc2a-2eba-438a-bd6b-ff26a656e153 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:24.573100 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1abc2a-2eba-438a-bd6b-ff26a656e153 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:24.574115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1abc2a-2eba-438a-bd6b-ff26a656e153 None admin] RBAC: Adding query filter params (role.id=a048041c27f44edf98a110d23138a327, scope.project.id=e87c7eebde3941f1aa5f40ce794e8687, user.id=381eba1a68b54e2eabaf4f64a95e41cd) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:24.574708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1abc2a-2eba-438a-bd6b-ff26a656e153 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:24.581582 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1abc2a-2eba-438a-bd6b-ff26a656e153 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:24.583662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 132/268] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:24 2022] GET /identity/v3/role_assignments?user.id=381eba1a68b54e2eabaf4f64a95e41cd&scope.project.id=e87c7eebde3941f1aa5f40ce794e8687&role.id=a048041c27f44edf98a110d23138a327 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:24.589230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0d363-711b-409f-be67-48521ce1ae05 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:24.589518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0d363-711b-409f-be67-48521ce1ae05 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:24.589794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0d363-711b-409f-be67-48521ce1ae05 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:24.591215 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 133/269] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:24.591947 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0d363-711b-409f-be67-48521ce1ae05 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:24.665151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 137/270] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:24.676295 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a0147-946f-4cc1-acc4-82333e91af6a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:24.676925 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a0147-946f-4cc1-acc4-82333e91af6a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:24.676925 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a0147-946f-4cc1-acc4-82333e91af6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:24.714023 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-622a0147-946f-4cc1-acc4-82333e91af6a None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:24.715268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-622a0147-946f-4cc1-acc4-82333e91af6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:24.815720 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e0d363-711b-409f-be67-48521ce1ae05 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:24.826166 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 138/271] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:24.826870 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a0147-946f-4cc1-acc4-82333e91af6a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:24.859990 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043a97d-4619-4c15-ab06-caaf4b16aa23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:24.861301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043a97d-4619-4c15-ab06-caaf4b16aa23 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:24.861884 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043a97d-4619-4c15-ab06-caaf4b16aa23 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:24.862354 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043a97d-4619-4c15-ab06-caaf4b16aa23 None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:24.866852 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043a97d-4619-4c15-ab06-caaf4b16aa23 None admin] RBAC: Authorizing `identity:get_project(project_id=e87c7eebde3941f1aa5f40ce794e8687)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:24.869012 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043a97d-4619-4c15-ab06-caaf4b16aa23 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:24.871196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 134/272] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:24 2022] GET /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:24.872190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043a97d-4619-4c15-ab06-caaf4b16aa23 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:24.893530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:24.900197 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:24.966844 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a0147-946f-4cc1-acc4-82333e91af6a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:25.009737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043a97d-4619-4c15-ab06-caaf4b16aa23 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:25.012405 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3c500-e4b3-469c-973d-86255c0a7801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:25.013221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3c500-e4b3-469c-973d-86255c0a7801 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:25.013510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3c500-e4b3-469c-973d-86255c0a7801 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:25.014487 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3c500-e4b3-469c-973d-86255c0a7801 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:25.020897 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a3c500-e4b3-469c-973d-86255c0a7801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:25.022060 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a3c500-e4b3-469c-973d-86255c0a7801 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:25.024254 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 139/273] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:25.025158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3c500-e4b3-469c-973d-86255c0a7801 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:25.059025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbe187-93b6-488f-8f96-645528a9e893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:25.059835 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbe187-93b6-488f-8f96-645528a9e893 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:25.060107 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbe187-93b6-488f-8f96-645528a9e893 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:25.060386 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbe187-93b6-488f-8f96-645528a9e893 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:25.071939 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dbe187-93b6-488f-8f96-645528a9e893 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:25.079636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-c6dbe187-93b6-488f-8f96-645528a9e893 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:25.079636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:25.079636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:25.079636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:25.079636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:25.079636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:25.079636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:25.079636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:25.079636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:25.079636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:25.079636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:25.079636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:25.079636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:25.079636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:25.079636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:25.079636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:25.080524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:25.080524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:25.080524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:25.080524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:25.080524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:25.080524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:25.080524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:25.080524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:25.080524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:25.080524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:25.080524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:25.080524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:25.080524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:25.080524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:25.080524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:25.080524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:25.081319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:25.081319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:25.081319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:25.081319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:25.081319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:25.081319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:25.081319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:25.081319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:25.081319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:25.081319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:25.081319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:25.081319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:25.081319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 135/274] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:25.083160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd107aa-7675-4b1e-acce-166d00294e57 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:25.083160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd107aa-7675-4b1e-acce-166d00294e57 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:25.083334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd107aa-7675-4b1e-acce-166d00294e57 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:25.086974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 136/275] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:25.086974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd107aa-7675-4b1e-acce-166d00294e57 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:25.148251 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3c500-e4b3-469c-973d-86255c0a7801 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:25.161919 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:25.165252 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:25.182401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69bdc4-5d63-4b3a-b0c9-a08c5245398e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:25.183136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69bdc4-5d63-4b3a-b0c9-a08c5245398e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:25.183419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69bdc4-5d63-4b3a-b0c9-a08c5245398e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:25.183702 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69bdc4-5d63-4b3a-b0c9-a08c5245398e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:25.184559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69bdc4-5d63-4b3a-b0c9-a08c5245398e None admin] RBAC: Adding query filter params (role.id=f0f7e28cbe314921afdfb657b163436f, scope.project.id=e87c7eebde3941f1aa5f40ce794e8687, user.id=381eba1a68b54e2eabaf4f64a95e41cd) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:25.184832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69bdc4-5d63-4b3a-b0c9-a08c5245398e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:25.186501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69bdc4-5d63-4b3a-b0c9-a08c5245398e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:25.190840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd107aa-7675-4b1e-acce-166d00294e57 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:25.198600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 140/276] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:25 2022] GET /identity/v3/role_assignments?user.id=381eba1a68b54e2eabaf4f64a95e41cd&scope.project.id=e87c7eebde3941f1aa5f40ce794e8687&role.id=f0f7e28cbe314921afdfb657b163436f => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:25.198600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69bdc4-5d63-4b3a-b0c9-a08c5245398e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:25.244108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd1dec-510a-48c7-b403-4bb03ef862c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:25.244828 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd1dec-510a-48c7-b403-4bb03ef862c3 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:25.245127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd1dec-510a-48c7-b403-4bb03ef862c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:25.245473 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd1dec-510a-48c7-b403-4bb03ef862c3 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:25.246217 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd1dec-510a-48c7-b403-4bb03ef862c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:25.247799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdd1dec-510a-48c7-b403-4bb03ef862c3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:25.319937 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69bdc4-5d63-4b3a-b0c9-a08c5245398e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:25.359838 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8716a6d-6a9d-43dd-b046-3dcb9ef7f306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:25.360575 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8716a6d-6a9d-43dd-b046-3dcb9ef7f306 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:25.360863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8716a6d-6a9d-43dd-b046-3dcb9ef7f306 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:25.361195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8716a6d-6a9d-43dd-b046-3dcb9ef7f306 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:25.361825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 137/277] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jun 24 03:58:25 2022] POST /identity/v3/users => generated 293 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:58:25.362500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8716a6d-6a9d-43dd-b046-3dcb9ef7f306 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:25.362816 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8716a6d-6a9d-43dd-b046-3dcb9ef7f306 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:25.365702 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8716a6d-6a9d-43dd-b046-3dcb9ef7f306 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:25.367680 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a7305-8bbf-4c6e-9e9b-c67dbe29180d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:25.367977 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a7305-8bbf-4c6e-9e9b-c67dbe29180d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:25.368270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a7305-8bbf-4c6e-9e9b-c67dbe29180d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:25.381733 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 141/278] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:25.383203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8716a6d-6a9d-43dd-b046-3dcb9ef7f306 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:25.401583 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-437a7305-8bbf-4c6e-9e9b-c67dbe29180d None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:25.402992 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-437a7305-8bbf-4c6e-9e9b-c67dbe29180d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:25.448162 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:25.501962 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 138/279] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:25.525998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8716a6d-6a9d-43dd-b046-3dcb9ef7f306 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:25.533878 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a7305-8bbf-4c6e-9e9b-c67dbe29180d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:25.575532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df612d-ed70-42f8-8bff-a29fedfcb938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:25.576482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df612d-ed70-42f8-8bff-a29fedfcb938 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:25.576792 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df612d-ed70-42f8-8bff-a29fedfcb938 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:25.577103 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df612d-ed70-42f8-8bff-a29fedfcb938 None admin] PATH_INFO: `/v3/users/4ff482e97e4544cca0ef08eca8fa6254` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:25.595565 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df612d-ed70-42f8-8bff-a29fedfcb938 None admin] RBAC: Authorizing `identity:get_user(user_id=4ff482e97e4544cca0ef08eca8fa6254)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:25.597217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df612d-ed70-42f8-8bff-a29fedfcb938 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:25.602493 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 142/280] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:25 2022] GET /identity/v3/users/4ff482e97e4544cca0ef08eca8fa6254 => generated 293 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:25.605075 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df612d-ed70-42f8-8bff-a29fedfcb938 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:25.610444 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:25.665809 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a7305-8bbf-4c6e-9e9b-c67dbe29180d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:25.713298 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f694e2-17ed-40e3-b9e7-19ff25c84cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:25.714832 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f694e2-17ed-40e3-b9e7-19ff25c84cfd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:25.715448 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f694e2-17ed-40e3-b9e7-19ff25c84cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:25.715728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f694e2-17ed-40e3-b9e7-19ff25c84cfd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:25.729276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f694e2-17ed-40e3-b9e7-19ff25c84cfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:25.739941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-52f694e2-17ed-40e3-b9e7-19ff25c84cfd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 03:58:25.739941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:25.739941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:25.739941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:25.739941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:25.739941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:25.739941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:25.739941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:25.739941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:25.739941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:25.739941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:25.739941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:25.739941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:25.739941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:25.739941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:25.739941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:25.740908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:25.740908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:25.740908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:25.740908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:25.740908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:25.740908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:25.740908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:25.740908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:25.740908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:25.740908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:25.740908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:25.740908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:25.740908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:25.740908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:25.740908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:25.740908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:25.741737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:25.741737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:25.741737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:25.741737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:25.741737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:25.741737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:25.741737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:25.741737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:25.741737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:25.741737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:25.741737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 03:58:25.741737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:25.742430 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 139/281] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:58:25.747933 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df612d-ed70-42f8-8bff-a29fedfcb938 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:25.749694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:25.754594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f694e2-17ed-40e3-b9e7-19ff25c84cfd None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:25.794028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577e951-a030-4db8-8762-4e929e92b29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:25.795281 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577e951-a030-4db8-8762-4e929e92b29d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:25.796609 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577e951-a030-4db8-8762-4e929e92b29d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:25.797141 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577e951-a030-4db8-8762-4e929e92b29d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:25.798325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577e951-a030-4db8-8762-4e929e92b29d None admin] RBAC: Adding query filter params (role.id=d7aadc20cd4c4838ae5d7d8e8826abae, user.id=4ff482e97e4544cca0ef08eca8fa6254, scope.system=all) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:25.802238 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577e951-a030-4db8-8762-4e929e92b29d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:25.802238 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577e951-a030-4db8-8762-4e929e92b29d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:25.808771 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 143/282] 10.222.0.16 () {56 vars in 1308 bytes} [Fri Jun 24 03:58:25 2022] GET /identity/v3/role_assignments?user.id=4ff482e97e4544cca0ef08eca8fa6254&scope.system=all&role.id=d7aadc20cd4c4838ae5d7d8e8826abae => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:25.865782 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c99e3a-ad29-4ee5-b1c6-e459d43ec648 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:25.866444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c99e3a-ad29-4ee5-b1c6-e459d43ec648 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:25.866909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c99e3a-ad29-4ee5-b1c6-e459d43ec648 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:25.869353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 144/283] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 03:58:25.875142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818b326-a763-4e89-b3c8-def0153be6ee None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:25.876368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818b326-a763-4e89-b3c8-def0153be6ee None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:25.877044 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818b326-a763-4e89-b3c8-def0153be6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:25.888970 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f694e2-17ed-40e3-b9e7-19ff25c84cfd None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:25.902330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b818b326-a763-4e89-b3c8-def0153be6ee None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:25.904220 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b818b326-a763-4e89-b3c8-def0153be6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:25.916390 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:25.933481 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbaf0db-8be4-4fcf-b84c-78b93bedc155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:25.934334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbaf0db-8be4-4fcf-b84c-78b93bedc155 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:25.934737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbaf0db-8be4-4fcf-b84c-78b93bedc155 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:25.935088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbaf0db-8be4-4fcf-b84c-78b93bedc155 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:25.935970 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbaf0db-8be4-4fcf-b84c-78b93bedc155 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:25.936358 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbaf0db-8be4-4fcf-b84c-78b93bedc155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:25.937879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbaf0db-8be4-4fcf-b84c-78b93bedc155 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:25.956768 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 140/284] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:25.962597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbaf0db-8be4-4fcf-b84c-78b93bedc155 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:26.038327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 145/285] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:26.064018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818b326-a763-4e89-b3c8-def0153be6ee None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:26.107143 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbaf0db-8be4-4fcf-b84c-78b93bedc155 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:26.181389 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71056c-b3e6-4b80-87bd-1d8858f5a3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:26.182274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71056c-b3e6-4b80-87bd-1d8858f5a3aa None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:26.182583 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71056c-b3e6-4b80-87bd-1d8858f5a3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:26.182862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71056c-b3e6-4b80-87bd-1d8858f5a3aa None admin] PATH_INFO: `/v3/users/c7419dda6a87453486513e85344eac1f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:26.187262 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71056c-b3e6-4b80-87bd-1d8858f5a3aa None admin] RBAC: Authorizing `identity:get_user(user_id=c7419dda6a87453486513e85344eac1f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:26.188752 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71056c-b3e6-4b80-87bd-1d8858f5a3aa None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:26.190458 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 141/286] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:25 2022] GET /identity/v3/users/c7419dda6a87453486513e85344eac1f => generated 297 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:26.196157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71056c-b3e6-4b80-87bd-1d8858f5a3aa None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:26.250288 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:26.265486 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818b326-a763-4e89-b3c8-def0153be6ee None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:26.300250 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:26.344303 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c06694-7969-48c6-a0b5-298da36b4128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:26.345109 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c06694-7969-48c6-a0b5-298da36b4128 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:26.345391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c06694-7969-48c6-a0b5-298da36b4128 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:26.345665 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c06694-7969-48c6-a0b5-298da36b4128 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:26.365369 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c06694-7969-48c6-a0b5-298da36b4128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:26.373408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-26c06694-7969-48c6-a0b5-298da36b4128 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:26.373408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:26.373408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:26.373408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:26.373408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:26.373408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:26.373408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:26.373408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:26.373408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:26.373408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:26.373408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:26.373408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:26.373408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:26.373408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:26.373408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:26.373408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:26.374317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:26.374317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:26.374317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:26.374317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:26.374317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:26.374317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:26.374317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:26.374317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:26.374317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:26.374317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:26.374317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:26.374317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:26.374317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:26.374317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:26.374317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:26.374317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:26.375142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:26.375142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:26.375142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:26.375142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:26.375142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:26.375142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:26.375142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:26.375142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:26.375142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:26.375142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:26.375142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:26.375142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:26.376437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 146/287] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:58:26.384859 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c06694-7969-48c6-a0b5-298da36b4128 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:26.402175 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71056c-b3e6-4b80-87bd-1d8858f5a3aa None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:26.463723 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed5af8-cb26-465c-b5ee-92c5ac72fcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:26.464952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed5af8-cb26-465c-b5ee-92c5ac72fcc0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:26.465463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed5af8-cb26-465c-b5ee-92c5ac72fcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:26.465950 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed5af8-cb26-465c-b5ee-92c5ac72fcc0 None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:26.467416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed5af8-cb26-465c-b5ee-92c5ac72fcc0 None admin] RBAC: Authorizing `identity:get_project(project_id=e87c7eebde3941f1aa5f40ce794e8687)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:26.469121 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bed5af8-cb26-465c-b5ee-92c5ac72fcc0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:26.471438 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 142/288] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:26 2022] GET /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:26.477838 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed5af8-cb26-465c-b5ee-92c5ac72fcc0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:26.514121 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c06694-7969-48c6-a0b5-298da36b4128 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:26.548281 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c151cb0-7bf2-40bf-9ae2-ba7667d1e8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:26.549001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c151cb0-7bf2-40bf-9ae2-ba7667d1e8fc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:26.549304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c151cb0-7bf2-40bf-9ae2-ba7667d1e8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:26.549704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c151cb0-7bf2-40bf-9ae2-ba7667d1e8fc None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:26.550433 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c151cb0-7bf2-40bf-9ae2-ba7667d1e8fc None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:26.550761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c151cb0-7bf2-40bf-9ae2-ba7667d1e8fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:26.552531 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c151cb0-7bf2-40bf-9ae2-ba7667d1e8fc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:26.564314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 147/289] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:26.567254 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c151cb0-7bf2-40bf-9ae2-ba7667d1e8fc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:26.594758 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:26.607482 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed5af8-cb26-465c-b5ee-92c5ac72fcc0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:26.631557 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:26.647656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa26145-c29d-43d9-be4b-3046272ae331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:26.648726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa26145-c29d-43d9-be4b-3046272ae331 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:26.649007 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa26145-c29d-43d9-be4b-3046272ae331 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:26.649279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa26145-c29d-43d9-be4b-3046272ae331 None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687/users/c7419dda6a87453486513e85344eac1f/roles/368ae945e96746faa1dc2ecbcefcf0f1` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:26.651325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa26145-c29d-43d9-be4b-3046272ae331 None admin] RBAC: Authorizing `identity:create_grant(project_id=e87c7eebde3941f1aa5f40ce794e8687, user_id=c7419dda6a87453486513e85344eac1f, role_id=368ae945e96746faa1dc2ecbcefcf0f1)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:26.653476 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa26145-c29d-43d9-be4b-3046272ae331 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:26.719497 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c151cb0-7bf2-40bf-9ae2-ba7667d1e8fc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:26.742105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 143/290] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 03:58:26 2022] PUT /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687/users/c7419dda6a87453486513e85344eac1f/roles/368ae945e96746faa1dc2ecbcefcf0f1 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 03:58:26.764612 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a18c17-68a7-4375-bc33-b962e2a842a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:26.765507 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a18c17-68a7-4375-bc33-b962e2a842a9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:26.765792 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a18c17-68a7-4375-bc33-b962e2a842a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:26.766065 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a18c17-68a7-4375-bc33-b962e2a842a9 None admin] PATH_INFO: `/v3/users/0f4b38c1503d472195344d43a909a5df` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:26.767400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a18c17-68a7-4375-bc33-b962e2a842a9 None admin] RBAC: Authorizing `identity:get_user(user_id=0f4b38c1503d472195344d43a909a5df)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:26.768918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a18c17-68a7-4375-bc33-b962e2a842a9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:26.771831 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 148/291] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:26 2022] GET /identity/v3/users/0f4b38c1503d472195344d43a909a5df => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:26.777235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa26145-c29d-43d9-be4b-3046272ae331 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:26.915125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa26145-c29d-43d9-be4b-3046272ae331 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:26.955845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300db2e-3423-40f3-8886-b4a093a7a433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:26.957438 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300db2e-3423-40f3-8886-b4a093a7a433 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:26.957991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300db2e-3423-40f3-8886-b4a093a7a433 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:26.958522 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300db2e-3423-40f3-8886-b4a093a7a433 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:27.057853 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:27.082558 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300db2e-3423-40f3-8886-b4a093a7a433 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:27.084655 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300db2e-3423-40f3-8886-b4a093a7a433 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:27.197992 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 144/292] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:27.203066 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a18c17-68a7-4375-bc33-b962e2a842a9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:27.296602 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:27.315853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a18c17-68a7-4375-bc33-b962e2a842a9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:27.353857 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c42b6c-f605-44b6-88d7-ff135329c917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:27.354780 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c42b6c-f605-44b6-88d7-ff135329c917 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:27.355065 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c42b6c-f605-44b6-88d7-ff135329c917 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:27.355343 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c42b6c-f605-44b6-88d7-ff135329c917 None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:27.356539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c42b6c-f605-44b6-88d7-ff135329c917 None admin] RBAC: Authorizing `identity:get_project(project_id=e87c7eebde3941f1aa5f40ce794e8687)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:27.357976 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c42b6c-f605-44b6-88d7-ff135329c917 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:27.360010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 149/293] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:27 2022] GET /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:27.365785 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300db2e-3423-40f3-8886-b4a093a7a433 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:27.489082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:27.521605 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300db2e-3423-40f3-8886-b4a093a7a433 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:27.571433 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd2548a-717d-4b9f-bdeb-2b6b52a1b9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:27.572478 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2548a-717d-4b9f-bdeb-2b6b52a1b9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:27.572967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2548a-717d-4b9f-bdeb-2b6b52a1b9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:27.573455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2548a-717d-4b9f-bdeb-2b6b52a1b9d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:27.574547 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2548a-717d-4b9f-bdeb-2b6b52a1b9d0 None admin] RBAC: Adding query filter params (role.id=d7aadc20cd4c4838ae5d7d8e8826abae, scope.project.id=e87c7eebde3941f1aa5f40ce794e8687, user.id=0f4b38c1503d472195344d43a909a5df) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:27.575099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2548a-717d-4b9f-bdeb-2b6b52a1b9d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:27.576754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd2548a-717d-4b9f-bdeb-2b6b52a1b9d0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:27.591873 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 145/294] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:27 2022] GET /identity/v3/role_assignments?user.id=0f4b38c1503d472195344d43a909a5df&scope.project.id=e87c7eebde3941f1aa5f40ce794e8687&role.id=d7aadc20cd4c4838ae5d7d8e8826abae => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:27.974473 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9a237-3169-42b0-b692-0212f48eaa8d None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:27.975020 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9a237-3169-42b0-b692-0212f48eaa8d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:27.975020 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9a237-3169-42b0-b692-0212f48eaa8d None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:27.976665 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 150/295] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:27.982169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1270605-a563-4d8f-aa0a-e79da6484454 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:27.983036 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1270605-a563-4d8f-aa0a-e79da6484454 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:27.983542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1270605-a563-4d8f-aa0a-e79da6484454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:28.049121 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f1270605-a563-4d8f-aa0a-e79da6484454 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:28.050603 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f1270605-a563-4d8f-aa0a-e79da6484454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:28.137896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 146/296] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:28.148381 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb297439-6481-4bec-bcf7-b3905900f3f0 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:28.148676 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb297439-6481-4bec-bcf7-b3905900f3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:28.148960 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb297439-6481-4bec-bcf7-b3905900f3f0 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:28.151501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 151/297] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:28.155628 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da7a65-61a1-4961-9031-e5c7d552f658 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:28.155926 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da7a65-61a1-4961-9031-e5c7d552f658 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:28.156212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da7a65-61a1-4961-9031-e5c7d552f658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:28.162045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb297439-6481-4bec-bcf7-b3905900f3f0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:28.173456 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a8da7a65-61a1-4961-9031-e5c7d552f658 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:28.174691 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a8da7a65-61a1-4961-9031-e5c7d552f658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:28.266613 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 147/298] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:28.282928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb297439-6481-4bec-bcf7-b3905900f3f0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:28.290295 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da7a65-61a1-4961-9031-e5c7d552f658 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:28.292452 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:28.326746 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dff3ad-6382-4f15-9255-dd5108434a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:28.327974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dff3ad-6382-4f15-9255-dd5108434a9c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:28.328534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dff3ad-6382-4f15-9255-dd5108434a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:28.329850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dff3ad-6382-4f15-9255-dd5108434a9c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:28.341419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dff3ad-6382-4f15-9255-dd5108434a9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:28.350093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-c0dff3ad-6382-4f15-9255-dd5108434a9c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 03:58:28.350093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:28.350093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:28.350093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:28.350093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:28.350093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:28.350093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:28.350093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:28.350093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:28.350093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:28.350093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:28.350093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:28.350093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:28.350093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:28.350093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:28.350093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:28.351041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:28.351041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:28.351041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:28.351041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:28.351041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:28.351041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:28.351041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:28.351041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:28.351041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:28.351041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:28.351041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:28.351041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:28.351041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:28.351041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:28.351041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:28.351041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:28.351820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:28.351820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:28.351820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:28.351820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:28.351820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:28.351820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:28.351820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:28.351820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:28.351820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:28.351820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:28.351820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 03:58:28.351820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:28.354600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 152/299] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:28.371340 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dff3ad-6382-4f15-9255-dd5108434a9c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:28.391784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:28.408086 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da7a65-61a1-4961-9031-e5c7d552f658 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:28.449771 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c7d416-b0d2-414e-a06e-449bc876be10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:28.450705 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7d416-b0d2-414e-a06e-449bc876be10 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:28.451072 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7d416-b0d2-414e-a06e-449bc876be10 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:28.451425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7d416-b0d2-414e-a06e-449bc876be10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:28.452779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c7d416-b0d2-414e-a06e-449bc876be10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:28.454458 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c7d416-b0d2-414e-a06e-449bc876be10 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:28.456280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 148/300] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:28.462416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c7d416-b0d2-414e-a06e-449bc876be10 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:28.501696 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dff3ad-6382-4f15-9255-dd5108434a9c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:28.536551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db57ff-a100-43a9-ab9a-be60443b26c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:28.537165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db57ff-a100-43a9-ab9a-be60443b26c7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:28.537456 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db57ff-a100-43a9-ab9a-be60443b26c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:28.537742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db57ff-a100-43a9-ab9a-be60443b26c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:28.538546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db57ff-a100-43a9-ab9a-be60443b26c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:28.539082 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db57ff-a100-43a9-ab9a-be60443b26c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:28.541077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db57ff-a100-43a9-ab9a-be60443b26c7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:28.544246 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:28.552187 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 153/301] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:28.559773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e7dfd-64c4-46ef-a65e-23e59923101f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:28.559773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e7dfd-64c4-46ef-a65e-23e59923101f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:28.559773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e7dfd-64c4-46ef-a65e-23e59923101f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:28.559773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 154/302] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:28.559773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e7dfd-64c4-46ef-a65e-23e59923101f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:28.583956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c7d416-b0d2-414e-a06e-449bc876be10 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:28.624531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fbd57-572d-48d6-8227-f4c038b01092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:28.625204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fbd57-572d-48d6-8227-f4c038b01092 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:28.625491 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fbd57-572d-48d6-8227-f4c038b01092 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:28.625772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fbd57-572d-48d6-8227-f4c038b01092 None admin] PATH_INFO: `/v3/groups` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:28.626506 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21fbd57-572d-48d6-8227-f4c038b01092 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:28.627954 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21fbd57-572d-48d6-8227-f4c038b01092 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:28.636245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:28.666809 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e7dfd-64c4-46ef-a65e-23e59923101f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:28.699739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8aac9-2a07-4f2b-86b8-e5470b3b5733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:28.700614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8aac9-2a07-4f2b-86b8-e5470b3b5733 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:28.700902 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8aac9-2a07-4f2b-86b8-e5470b3b5733 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:28.701181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8aac9-2a07-4f2b-86b8-e5470b3b5733 None admin] PATH_INFO: `/v3/users/283f10fa85394c54bc993fab80930303` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:28.715036 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8aac9-2a07-4f2b-86b8-e5470b3b5733 None admin] RBAC: Authorizing `identity:get_user(user_id=283f10fa85394c54bc993fab80930303)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:28.722700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8aac9-2a07-4f2b-86b8-e5470b3b5733 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:28.722700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 155/303] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:28 2022] GET /identity/v3/users/283f10fa85394c54bc993fab80930303 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:28.722700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af428cf6-0b88-4680-921f-21af20720775 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:28.722700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af428cf6-0b88-4680-921f-21af20720775 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:28.722700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af428cf6-0b88-4680-921f-21af20720775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:28.735681 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-af428cf6-0b88-4680-921f-21af20720775 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:28.736858 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-af428cf6-0b88-4680-921f-21af20720775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:28.753328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 149/304] 10.222.0.16 () {60 vars in 1134 bytes} [Fri Jun 24 03:58:28 2022] POST /identity/v3/groups => generated 229 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 03:58:28.755251 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fbd57-572d-48d6-8227-f4c038b01092 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:28.834795 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 156/305] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:28.857024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af428cf6-0b88-4680-921f-21af20720775 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:28.912732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fbd57-572d-48d6-8227-f4c038b01092 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:28.926137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:28.955547 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:28.966203 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398eb7bf-e808-438e-af71-64c42f5a5d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:28.967331 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398eb7bf-e808-438e-af71-64c42f5a5d78 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:28.967899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398eb7bf-e808-438e-af71-64c42f5a5d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:28.968193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398eb7bf-e808-438e-af71-64c42f5a5d78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:28.969255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398eb7bf-e808-438e-af71-64c42f5a5d78 None admin] RBAC: Adding query filter params (role.id=368ae945e96746faa1dc2ecbcefcf0f1, user.id=283f10fa85394c54bc993fab80930303, scope.system=all) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:28.969888 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398eb7bf-e808-438e-af71-64c42f5a5d78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:28.971421 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398eb7bf-e808-438e-af71-64c42f5a5d78 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:28.974325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af428cf6-0b88-4680-921f-21af20720775 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:28.978568 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 150/306] 10.222.0.16 () {56 vars in 1308 bytes} [Fri Jun 24 03:58:28 2022] GET /identity/v3/role_assignments?user.id=283f10fa85394c54bc993fab80930303&scope.system=all&role.id=368ae945e96746faa1dc2ecbcefcf0f1 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:29.019057 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885753be-8412-451a-b133-9308d3b36c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:29.020592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885753be-8412-451a-b133-9308d3b36c53 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:29.020889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885753be-8412-451a-b133-9308d3b36c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:29.021170 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885753be-8412-451a-b133-9308d3b36c53 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:29.033327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885753be-8412-451a-b133-9308d3b36c53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:29.042048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-885753be-8412-451a-b133-9308d3b36c53 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:29.042048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:29.042048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:29.042048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:29.042048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:29.042048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:29.042048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:29.042048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:29.042048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:29.042048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:29.042048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:29.042048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:29.042048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:29.042048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:29.042048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:29.042048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:29.043018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:29.043018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:29.043018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:29.043018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:29.043018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:29.043018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:29.043018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:29.043018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:29.043018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:29.043018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:29.043018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:29.043018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:29.043018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:29.043018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:29.043018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:29.043018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:29.044799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:29.044799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:29.044799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:29.044799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:29.044799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:29.044799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:29.044799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:29.044799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:29.044799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:29.044799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:29.044799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:29.044799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:29.044799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 157/307] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:58:29.049564 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398eb7bf-e808-438e-af71-64c42f5a5d78 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:29.166740 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398eb7bf-e808-438e-af71-64c42f5a5d78 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:29.202410 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34625f76-4be1-4cc7-ad52-eb9c7a9a6360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:29.203381 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34625f76-4be1-4cc7-ad52-eb9c7a9a6360 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:29.203881 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34625f76-4be1-4cc7-ad52-eb9c7a9a6360 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:29.204338 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34625f76-4be1-4cc7-ad52-eb9c7a9a6360 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:29.205276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34625f76-4be1-4cc7-ad52-eb9c7a9a6360 None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:29.205776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34625f76-4be1-4cc7-ad52-eb9c7a9a6360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:29.208044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34625f76-4be1-4cc7-ad52-eb9c7a9a6360 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:29.217554 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 151/308] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:29.223809 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885753be-8412-451a-b133-9308d3b36c53 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:29.241542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ededb-536e-40ee-80f7-e551c17c13e8 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:29.241833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ededb-536e-40ee-80f7-e551c17c13e8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:29.242114 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ededb-536e-40ee-80f7-e551c17c13e8 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:29.243926 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 152/309] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:29.248760 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725e601-3952-4625-8cf6-3a8b941a121a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:29.249060 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725e601-3952-4625-8cf6-3a8b941a121a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:29.249343 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725e601-3952-4625-8cf6-3a8b941a121a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:29.266241 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e725e601-3952-4625-8cf6-3a8b941a121a None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:29.267484 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e725e601-3952-4625-8cf6-3a8b941a121a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:29.327675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:29.348178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 153/310] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:58:29.361350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885753be-8412-451a-b133-9308d3b36c53 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:29.372385 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725e601-3952-4625-8cf6-3a8b941a121a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:29.385942 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:29.406825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e2218-8664-40e0-a6f3-39895a0f0cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:29.408010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e2218-8664-40e0-a6f3-39895a0f0cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:29.408157 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e2218-8664-40e0-a6f3-39895a0f0cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:29.408449 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e2218-8664-40e0-a6f3-39895a0f0cb9 None admin] PATH_INFO: `/v3/users/4ff482e97e4544cca0ef08eca8fa6254` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:29.409665 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e2218-8664-40e0-a6f3-39895a0f0cb9 None admin] RBAC: Authorizing `identity:get_user(user_id=4ff482e97e4544cca0ef08eca8fa6254)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:29.411500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e2218-8664-40e0-a6f3-39895a0f0cb9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:29.413656 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 158/311] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:29 2022] GET /identity/v3/users/4ff482e97e4544cca0ef08eca8fa6254 => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:29.419011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e2218-8664-40e0-a6f3-39895a0f0cb9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:29.503254 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725e601-3952-4625-8cf6-3a8b941a121a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:29.528347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e2218-8664-40e0-a6f3-39895a0f0cb9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:29.544099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e1371-b881-4daf-af09-d2c435ddb282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:29.545062 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e1371-b881-4daf-af09-d2c435ddb282 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:29.545419 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e1371-b881-4daf-af09-d2c435ddb282 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:29.545771 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e1371-b881-4daf-af09-d2c435ddb282 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:29.557134 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e1371-b881-4daf-af09-d2c435ddb282 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:29.565090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-f52e1371-b881-4daf-af09-d2c435ddb282 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 03:58:29.565090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:29.565090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:29.565090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:29.565090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:29.565090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:29.565090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:29.565090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:29.565090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:29.565090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:29.565090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:29.565090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:29.565090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:29.565090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:29.565090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:29.565090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:29.566712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:29.566712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:29.566712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:29.566712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:29.566712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:29.566712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:29.566712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:29.566712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:29.566712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:29.566712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:29.566712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:29.566712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:29.566712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:29.566712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:29.566712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:29.566712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:29.567747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:29.567747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:29.567747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:29.567747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:29.567747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:29.567747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:29.567747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:29.567747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:29.567747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:29.567747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:29.567747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 03:58:29.567747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:29.573610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 154/312] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:58:29.574654 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e1371-b881-4daf-af09-d2c435ddb282 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:29.575360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819c68c-e378-48fb-bdff-eb29466a0374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:29.576420 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819c68c-e378-48fb-bdff-eb29466a0374 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:29.576861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819c68c-e378-48fb-bdff-eb29466a0374 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:29.577293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819c68c-e378-48fb-bdff-eb29466a0374 None admin] PATH_INFO: `/v3/system/users/4ff482e97e4544cca0ef08eca8fa6254/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:29.580227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819c68c-e378-48fb-bdff-eb29466a0374 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4ff482e97e4544cca0ef08eca8fa6254, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:29.581891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819c68c-e378-48fb-bdff-eb29466a0374 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:29.625678 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:29.679816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e1371-b881-4daf-af09-d2c435ddb282 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:29.714268 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd467fe-cd10-4f38-93e3-48abe793b23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:29.714977 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd467fe-cd10-4f38-93e3-48abe793b23d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:29.715267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd467fe-cd10-4f38-93e3-48abe793b23d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:29.715550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd467fe-cd10-4f38-93e3-48abe793b23d None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:29.716300 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd467fe-cd10-4f38-93e3-48abe793b23d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:29.716615 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd467fe-cd10-4f38-93e3-48abe793b23d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:29.719392 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd467fe-cd10-4f38-93e3-48abe793b23d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:29.719501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 159/313] 10.222.0.16 () {58 vars in 1334 bytes} [Fri Jun 24 03:58:29 2022] PUT /identity/v3/system/users/4ff482e97e4544cca0ef08eca8fa6254/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:58:29.742812 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 155/314] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:29.748808 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819c68c-e378-48fb-bdff-eb29466a0374 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:29.809951 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:29.875504 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819c68c-e378-48fb-bdff-eb29466a0374 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:29.912400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349430b-e968-4a66-8dd3-24b93fa4c4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:29.913765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349430b-e968-4a66-8dd3-24b93fa4c4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:29.914199 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349430b-e968-4a66-8dd3-24b93fa4c4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:29.914519 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349430b-e968-4a66-8dd3-24b93fa4c4a4 None admin] PATH_INFO: `/v3/users/c7419dda6a87453486513e85344eac1f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:29.915825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349430b-e968-4a66-8dd3-24b93fa4c4a4 None admin] RBAC: Authorizing `identity:get_user(user_id=c7419dda6a87453486513e85344eac1f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:29.917740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349430b-e968-4a66-8dd3-24b93fa4c4a4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:29.919795 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 160/315] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:29 2022] GET /identity/v3/users/c7419dda6a87453486513e85344eac1f => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:29.925073 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd467fe-cd10-4f38-93e3-48abe793b23d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:30.049676 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd467fe-cd10-4f38-93e3-48abe793b23d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:30.060577 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:30.088009 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbbfb2-69e4-44b3-94ab-df1b59b3f650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:30.088661 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbbfb2-69e4-44b3-94ab-df1b59b3f650 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:30.088944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbbfb2-69e4-44b3-94ab-df1b59b3f650 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:30.089222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbbfb2-69e4-44b3-94ab-df1b59b3f650 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:30.165261 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dbbfb2-69e4-44b3-94ab-df1b59b3f650 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:30.166776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dbbfb2-69e4-44b3-94ab-df1b59b3f650 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:30.276715 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 156/316] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:30.282189 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349430b-e968-4a66-8dd3-24b93fa4c4a4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:30.387511 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349430b-e968-4a66-8dd3-24b93fa4c4a4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:30.411179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:30.424250 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540963a-f1e9-42ca-b04d-9b219b4812a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:30.425400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540963a-f1e9-42ca-b04d-9b219b4812a3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:30.426479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540963a-f1e9-42ca-b04d-9b219b4812a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:30.426770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540963a-f1e9-42ca-b04d-9b219b4812a3 None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:30.427993 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540963a-f1e9-42ca-b04d-9b219b4812a3 None admin] RBAC: Authorizing `identity:get_project(project_id=e87c7eebde3941f1aa5f40ce794e8687)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:30.429451 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540963a-f1e9-42ca-b04d-9b219b4812a3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:30.431262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 161/317] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:30 2022] GET /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:30.437251 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbbfb2-69e4-44b3-94ab-df1b59b3f650 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:30.448868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:30.552531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbbfb2-69e4-44b3-94ab-df1b59b3f650 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:30.588933 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fd1b8-293b-4f9d-8f10-8aa24f3f933e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:30.589960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fd1b8-293b-4f9d-8f10-8aa24f3f933e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:30.590493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fd1b8-293b-4f9d-8f10-8aa24f3f933e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:30.590966 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fd1b8-293b-4f9d-8f10-8aa24f3f933e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:30.591999 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642fd1b8-293b-4f9d-8f10-8aa24f3f933e None admin] RBAC: Adding query filter params (role.id=368ae945e96746faa1dc2ecbcefcf0f1, scope.project.id=e87c7eebde3941f1aa5f40ce794e8687, user.id=c7419dda6a87453486513e85344eac1f) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:30.592543 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642fd1b8-293b-4f9d-8f10-8aa24f3f933e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:30.594172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642fd1b8-293b-4f9d-8f10-8aa24f3f933e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:30.601249 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 157/318] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:30 2022] GET /identity/v3/role_assignments?user.id=c7419dda6a87453486513e85344eac1f&scope.project.id=e87c7eebde3941f1aa5f40ce794e8687&role.id=368ae945e96746faa1dc2ecbcefcf0f1 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:31.027347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a34d0e-c9d8-4413-9ede-3ef3edf6c0c4 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:31.027711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a34d0e-c9d8-4413-9ede-3ef3edf6c0c4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:31.027914 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a34d0e-c9d8-4413-9ede-3ef3edf6c0c4 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:31.032013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 162/319] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:31.034756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6d6a8-0e66-405a-8ca5-f3a8b87333de None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:31.035763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6d6a8-0e66-405a-8ca5-f3a8b87333de None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:31.036072 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6d6a8-0e66-405a-8ca5-f3a8b87333de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:31.053872 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1bc6d6a8-0e66-405a-8ca5-f3a8b87333de None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:31.055106 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1bc6d6a8-0e66-405a-8ca5-f3a8b87333de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:31.082233 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f767af-f26c-4ccd-950c-f362b303ba55 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:31.082572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f767af-f26c-4ccd-950c-f362b303ba55 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:31.082862 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f767af-f26c-4ccd-950c-f362b303ba55 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:31.084241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 163/320] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 03:58:31.089375 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892181b-1b86-43a9-b35f-dd29dfef0bf0 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:31.089662 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892181b-1b86-43a9-b35f-dd29dfef0bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:31.089941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892181b-1b86-43a9-b35f-dd29dfef0bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:31.107089 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6892181b-1b86-43a9-b35f-dd29dfef0bf0 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:31.108268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6892181b-1b86-43a9-b35f-dd29dfef0bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:31.132668 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 158/321] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:31.155661 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6d6a8-0e66-405a-8ca5-f3a8b87333de None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:31.158679 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:31.183669 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 164/322] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:58:31.203044 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892181b-1b86-43a9-b35f-dd29dfef0bf0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:31.266500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6d6a8-0e66-405a-8ca5-f3a8b87333de None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:31.304737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e6399-e910-420c-a964-0bbe383fb4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:31.305563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e6399-e910-420c-a964-0bbe383fb4c1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:31.305856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e6399-e910-420c-a964-0bbe383fb4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:31.306137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e6399-e910-420c-a964-0bbe383fb4c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:31.307343 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e6399-e910-420c-a964-0bbe383fb4c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:31.308808 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e6399-e910-420c-a964-0bbe383fb4c1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:31.310809 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 159/323] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:31.315678 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e6399-e910-420c-a964-0bbe383fb4c1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:31.320158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892181b-1b86-43a9-b35f-dd29dfef0bf0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:31.341123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:31.360985 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a12f47-b501-446b-8630-340425a1ec20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:31.362152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a12f47-b501-446b-8630-340425a1ec20 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:31.362475 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a12f47-b501-446b-8630-340425a1ec20 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:31.362757 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a12f47-b501-446b-8630-340425a1ec20 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:31.375943 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a12f47-b501-446b-8630-340425a1ec20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:31.383953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-a3a12f47-b501-446b-8630-340425a1ec20 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 03:58:31.383953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:31.383953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:31.383953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:31.383953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:31.383953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:31.383953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:31.383953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:31.383953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:31.383953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:31.383953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:31.383953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:31.383953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:31.383953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:31.383953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:31.383953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:31.384853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:31.384853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:31.384853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:31.384853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:31.384853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:31.384853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:31.384853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:31.384853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:31.384853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:31.384853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:31.384853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:31.384853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:31.384853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:31.384853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:31.384853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:31.384853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:31.385642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:31.385642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:31.385642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:31.385642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:31.385642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:31.385642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:31.385642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:31.385642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:31.385642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:31.385642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:31.385642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 03:58:31.385642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:31.385642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 165/324] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:58:31.391174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a12f47-b501-446b-8630-340425a1ec20 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:31.395298 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:31.430233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e6399-e910-420c-a964-0bbe383fb4c1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:31.467504 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56db57-e116-4207-90f4-d135666c6bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:31.468167 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56db57-e116-4207-90f4-d135666c6bcc None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:31.468446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56db57-e116-4207-90f4-d135666c6bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:31.468720 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56db57-e116-4207-90f4-d135666c6bcc None admin] PATH_INFO: `/v3/groups` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:31.469408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56db57-e116-4207-90f4-d135666c6bcc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:31.470933 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56db57-e116-4207-90f4-d135666c6bcc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:31.496800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a12f47-b501-446b-8630-340425a1ec20 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:31.529823 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e083fc3-e477-4f66-9b6d-7b0a9ff0b4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:31.530528 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e083fc3-e477-4f66-9b6d-7b0a9ff0b4bf None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:31.530812 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e083fc3-e477-4f66-9b6d-7b0a9ff0b4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:31.531089 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e083fc3-e477-4f66-9b6d-7b0a9ff0b4bf None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:31.531839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e083fc3-e477-4f66-9b6d-7b0a9ff0b4bf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:31.532147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e083fc3-e477-4f66-9b6d-7b0a9ff0b4bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:31.533551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e083fc3-e477-4f66-9b6d-7b0a9ff0b4bf None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:31.542596 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 166/325] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:31.547943 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e083fc3-e477-4f66-9b6d-7b0a9ff0b4bf None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:31.589796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:31.594534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 160/326] 10.222.0.16 () {60 vars in 1134 bytes} [Fri Jun 24 03:58:31 2022] POST /identity/v3/groups => generated 226 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 03:58:31.656145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e083fc3-e477-4f66-9b6d-7b0a9ff0b4bf None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:31.694157 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b31a2-1edc-462f-9669-849a0a4aa55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:31.695109 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b31a2-1edc-462f-9669-849a0a4aa55f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:31.695395 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b31a2-1edc-462f-9669-849a0a4aa55f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:31.695680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b31a2-1edc-462f-9669-849a0a4aa55f None admin] PATH_INFO: `/v3/users/283f10fa85394c54bc993fab80930303` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:31.696945 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b31a2-1edc-462f-9669-849a0a4aa55f None admin] RBAC: Authorizing `identity:get_user(user_id=283f10fa85394c54bc993fab80930303)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:31.698452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b31a2-1edc-462f-9669-849a0a4aa55f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:31.704824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 167/327] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:31 2022] GET /identity/v3/users/283f10fa85394c54bc993fab80930303 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:31.705912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56db57-e116-4207-90f4-d135666c6bcc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:31.812886 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56db57-e116-4207-90f4-d135666c6bcc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:31.830297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9aef4-1880-41ee-a62a-ed8fb83dda6c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:31.830631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9aef4-1880-41ee-a62a-ed8fb83dda6c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:31.830909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9aef4-1880-41ee-a62a-ed8fb83dda6c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:31.832536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 168/328] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:31.837777 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb7815-2c97-4353-a147-e47938158c8d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:31.838067 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb7815-2c97-4353-a147-e47938158c8d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:31.838528 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb7815-2c97-4353-a147-e47938158c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:31.845279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:31.852249 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab888e1-abb8-4f7f-81d8-0d5eda020070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:31.853142 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab888e1-abb8-4f7f-81d8-0d5eda020070 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:31.853497 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab888e1-abb8-4f7f-81d8-0d5eda020070 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:31.853784 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab888e1-abb8-4f7f-81d8-0d5eda020070 None admin] PATH_INFO: `/v3/system/users/283f10fa85394c54bc993fab80930303/roles/368ae945e96746faa1dc2ecbcefcf0f1` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:31.855376 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab888e1-abb8-4f7f-81d8-0d5eda020070 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=283f10fa85394c54bc993fab80930303, role_id=368ae945e96746faa1dc2ecbcefcf0f1)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:31.856984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab888e1-abb8-4f7f-81d8-0d5eda020070 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:31.859316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7eeb7815-2c97-4353-a147-e47938158c8d None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:31.860559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7eeb7815-2c97-4353-a147-e47938158c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:31.922764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 161/329] 10.222.0.16 () {58 vars in 1334 bytes} [Fri Jun 24 03:58:31 2022] PUT /identity/v3/system/users/283f10fa85394c54bc993fab80930303/roles/368ae945e96746faa1dc2ecbcefcf0f1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:58:31.939755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 169/330] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:58:31.964173 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab888e1-abb8-4f7f-81d8-0d5eda020070 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:32.098803 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab888e1-abb8-4f7f-81d8-0d5eda020070 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:32.138110 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747c2de-450a-4c2f-8a8d-094f10e0bf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:32.138957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747c2de-450a-4c2f-8a8d-094f10e0bf48 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:32.139245 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747c2de-450a-4c2f-8a8d-094f10e0bf48 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:32.139526 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747c2de-450a-4c2f-8a8d-094f10e0bf48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:32.153290 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747c2de-450a-4c2f-8a8d-094f10e0bf48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:32.162667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-5747c2de-450a-4c2f-8a8d-094f10e0bf48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:32.162667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:32.162667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:32.162667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:32.162667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:32.162667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:32.162667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:32.162667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:32.162667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:32.162667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:32.162667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:32.162667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:32.162667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:32.162667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:32.162667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:32.162667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:32.163576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:32.163576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:32.163576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:32.163576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:32.163576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:32.163576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:32.163576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:32.163576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:32.163576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:32.163576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:32.163576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:32.163576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:32.163576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:32.163576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:32.163576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:32.163576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:32.164373 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:32.164373 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:32.164373 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:32.164373 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:32.164373 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:32.164373 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:32.164373 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:32.164373 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:32.164373 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:32.164373 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:32.164373 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:32.164373 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:32.167141 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 162/331] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:32.177100 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb7815-2c97-4353-a147-e47938158c8d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:32.226586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:32.308129 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeb7815-2c97-4353-a147-e47938158c8d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:32.349576 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aaeee7-9154-4dff-b8cf-af9c7e0ca4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:32.350191 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaeee7-9154-4dff-b8cf-af9c7e0ca4b5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:32.350500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaeee7-9154-4dff-b8cf-af9c7e0ca4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:32.350782 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaeee7-9154-4dff-b8cf-af9c7e0ca4b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:32.351536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aaeee7-9154-4dff-b8cf-af9c7e0ca4b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:32.351850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aaeee7-9154-4dff-b8cf-af9c7e0ca4b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:32.353228 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aaeee7-9154-4dff-b8cf-af9c7e0ca4b5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:32.366463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 170/332] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:32.372516 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747c2de-450a-4c2f-8a8d-094f10e0bf48 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:32.443207 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:32.502353 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747c2de-450a-4c2f-8a8d-094f10e0bf48 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:32.544098 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0820299-ddc1-4240-acde-ff198bf70607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:32.555785 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0820299-ddc1-4240-acde-ff198bf70607 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:32.556314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0820299-ddc1-4240-acde-ff198bf70607 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:32.556821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0820299-ddc1-4240-acde-ff198bf70607 None admin] PATH_INFO: `/v3/users/4ff482e97e4544cca0ef08eca8fa6254` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:32.558413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0820299-ddc1-4240-acde-ff198bf70607 None admin] RBAC: Authorizing `identity:get_user(user_id=4ff482e97e4544cca0ef08eca8fa6254)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:32.560082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0820299-ddc1-4240-acde-ff198bf70607 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:32.562277 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 163/333] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:32 2022] GET /identity/v3/users/4ff482e97e4544cca0ef08eca8fa6254 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:32.569310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aaeee7-9154-4dff-b8cf-af9c7e0ca4b5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:32.737646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:32.744128 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aaeee7-9154-4dff-b8cf-af9c7e0ca4b5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:32.786433 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d35fdd-da5f-4312-856d-8d356cfa6238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:32.787113 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d35fdd-da5f-4312-856d-8d356cfa6238 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:32.787403 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d35fdd-da5f-4312-856d-8d356cfa6238 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:32.787687 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d35fdd-da5f-4312-856d-8d356cfa6238 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:32.788537 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d35fdd-da5f-4312-856d-8d356cfa6238 None admin] RBAC: Adding query filter params (role.id=d7aadc20cd4c4838ae5d7d8e8826abae, user.id=4ff482e97e4544cca0ef08eca8fa6254, scope.system=all) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:32.788856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d35fdd-da5f-4312-856d-8d356cfa6238 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:32.790311 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d35fdd-da5f-4312-856d-8d356cfa6238 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:32.803296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 171/334] 10.222.0.16 () {56 vars in 1308 bytes} [Fri Jun 24 03:58:32 2022] GET /identity/v3/role_assignments?user.id=4ff482e97e4544cca0ef08eca8fa6254&scope.system=all&role.id=d7aadc20cd4c4838ae5d7d8e8826abae => generated 510 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:34.386648 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec4523-4c85-40b0-a890-15fc516ac090 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:34.387627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec4523-4c85-40b0-a890-15fc516ac090 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:34.388128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec4523-4c85-40b0-a890-15fc516ac090 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:34.389905 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 164/335] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:34.396112 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe399e-1acc-4462-a25c-17458ac8213d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:34.396946 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe399e-1acc-4462-a25c-17458ac8213d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:34.397421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe399e-1acc-4462-a25c-17458ac8213d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:34.409296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47053a-1661-4a1c-9775-907e5ee154c6 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:34.409941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47053a-1661-4a1c-9775-907e5ee154c6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:34.411045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47053a-1661-4a1c-9775-907e5ee154c6 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:34.412627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 165/336] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 03:58:34.419740 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee675a3-c0dd-495b-b819-35d559c64244 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:34.419740 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee675a3-c0dd-495b-b819-35d559c64244 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:34.419740 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee675a3-c0dd-495b-b819-35d559c64244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:34.424368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-debe399e-1acc-4462-a25c-17458ac8213d None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:34.427062 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-debe399e-1acc-4462-a25c-17458ac8213d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:34.451612 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3ee675a3-c0dd-495b-b819-35d559c64244 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:34.452836 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3ee675a3-c0dd-495b-b819-35d559c64244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:34.581082 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 172/337] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:58:34.584685 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 166/338] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:34.621466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe399e-1acc-4462-a25c-17458ac8213d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:34.627026 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee675a3-c0dd-495b-b819-35d559c64244 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:34.726843 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:34.794774 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:34.807673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe399e-1acc-4462-a25c-17458ac8213d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:34.812145 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee675a3-c0dd-495b-b819-35d559c64244 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:34.858398 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f9b74-2d3d-490b-b4bb-14fdcae8b779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:34.859511 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f9b74-2d3d-490b-b4bb-14fdcae8b779 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:34.860005 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f9b74-2d3d-490b-b4bb-14fdcae8b779 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:34.860462 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f9b74-2d3d-490b-b4bb-14fdcae8b779 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:34.865277 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb5c46f-d3b7-459a-bc90-7bbd19629fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:34.865277 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5c46f-d3b7-459a-bc90-7bbd19629fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:34.865277 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5c46f-d3b7-459a-bc90-7bbd19629fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:34.865277 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb5c46f-d3b7-459a-bc90-7bbd19629fa2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:34.883847 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb5c46f-d3b7-459a-bc90-7bbd19629fa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:34.893781 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f9b74-2d3d-490b-b4bb-14fdcae8b779 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:34.903571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-3bb5c46f-d3b7-459a-bc90-7bbd19629fa2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:34.903571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:34.903571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:34.903571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:34.903571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:34.903571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:34.903571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:34.903571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:34.903571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:34.903571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:34.903571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:34.903571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:34.903571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:34.903571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:34.903571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:34.903571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:34.904493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:34.904493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:34.904493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:34.904493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:34.904493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:34.904493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:34.904493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:34.904493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:34.904493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:34.904493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:34.904493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:34.904493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:34.904493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:34.904493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:34.904493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:34.904493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:34.905288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:34.905288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:34.905288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:34.905288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:34.905288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:34.905288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:34.905288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:34.905288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:34.905288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:34.905288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:34.905288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:34.905288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:34.905288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 167/339] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:34.906478 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d81ced-81d3-4f4b-b14f-48bd48234345 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:34.906478 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d81ced-81d3-4f4b-b14f-48bd48234345 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:34.906721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d81ced-81d3-4f4b-b14f-48bd48234345 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:34.907293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-332f9b74-2d3d-490b-b4bb-14fdcae8b779 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 03:58:34.907293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:34.907293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:34.907293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:34.907293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:34.907293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:34.907293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:34.907293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:34.907293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:34.907293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:34.907293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:34.907293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:34.907293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:34.907293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:34.907293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:34.907293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:34.908183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:34.908183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:34.908183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:34.908183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:34.908183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:34.908183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:34.908183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:34.908183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:34.908183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:34.908183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:34.908183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:34.908183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:34.908183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:34.908183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:34.908183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:34.908183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:34.909029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:34.909029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:34.909029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:34.909029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:34.909029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:34.909029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:34.909029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:34.909029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:34.909029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:34.909029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:34.909029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 03:58:34.909029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:34.909635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 168/340] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:34.911539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 173/341] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:34.916201 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d81ced-81d3-4f4b-b14f-48bd48234345 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:34.917576 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f9b74-2d3d-490b-b4bb-14fdcae8b779 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:35.068587 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d81ced-81d3-4f4b-b14f-48bd48234345 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:35.090217 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:35.108710 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e0baf-7a2b-4c76-a754-0cd4b16885bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:35.109813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e0baf-7a2b-4c76-a754-0cd4b16885bd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:35.110335 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e0baf-7a2b-4c76-a754-0cd4b16885bd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:35.110891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e0baf-7a2b-4c76-a754-0cd4b16885bd None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:35.111943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e0baf-7a2b-4c76-a754-0cd4b16885bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:35.112554 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e0baf-7a2b-4c76-a754-0cd4b16885bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:35.114743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e0baf-7a2b-4c76-a754-0cd4b16885bd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:35.118955 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f9b74-2d3d-490b-b4bb-14fdcae8b779 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:35.126355 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 169/342] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:35.127909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c39083-090c-47c1-9df8-37607f5e8e93 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:35.128402 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c39083-090c-47c1-9df8-37607f5e8e93 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:35.128938 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c39083-090c-47c1-9df8-37607f5e8e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:35.146987 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a1c39083-090c-47c1-9df8-37607f5e8e93 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:35.148561 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a1c39083-090c-47c1-9df8-37607f5e8e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:35.188573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04683e6c-ca75-423e-96ed-434b87aee633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:35.189276 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04683e6c-ca75-423e-96ed-434b87aee633 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:35.189577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04683e6c-ca75-423e-96ed-434b87aee633 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:35.189861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04683e6c-ca75-423e-96ed-434b87aee633 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:35.191177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04683e6c-ca75-423e-96ed-434b87aee633 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:35.191177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04683e6c-ca75-423e-96ed-434b87aee633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:35.196072 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04683e6c-ca75-423e-96ed-434b87aee633 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:35.210230 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 174/343] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:35.213389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04683e6c-ca75-423e-96ed-434b87aee633 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:35.227047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 170/344] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:35.228198 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c39083-090c-47c1-9df8-37607f5e8e93 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:35.233307 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:35.335695 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c39083-090c-47c1-9df8-37607f5e8e93 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:35.343780 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:35.372490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66ab83-f647-41d5-9e60-651e5d1528a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:35.373914 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66ab83-f647-41d5-9e60-651e5d1528a9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:35.374462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66ab83-f647-41d5-9e60-651e5d1528a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:35.374945 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be66ab83-f647-41d5-9e60-651e5d1528a9 None admin] PATH_INFO: `/v3/users/283f10fa85394c54bc993fab80930303` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:35.376487 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66ab83-f647-41d5-9e60-651e5d1528a9 None admin] RBAC: Authorizing `identity:get_user(user_id=283f10fa85394c54bc993fab80930303)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:35.378235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be66ab83-f647-41d5-9e60-651e5d1528a9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:35.381162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 171/345] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 03:58:35 2022] GET /identity/v3/users/283f10fa85394c54bc993fab80930303 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:35.382122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66ab83-f647-41d5-9e60-651e5d1528a9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:35.388822 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04683e6c-ca75-423e-96ed-434b87aee633 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:35.443916 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef92cc9-37d5-4cff-a9c9-560b74d31730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:35.445051 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef92cc9-37d5-4cff-a9c9-560b74d31730 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:35.445337 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef92cc9-37d5-4cff-a9c9-560b74d31730 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:35.445845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef92cc9-37d5-4cff-a9c9-560b74d31730 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:35.493817 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be66ab83-f647-41d5-9e60-651e5d1528a9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:35.518409 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef92cc9-37d5-4cff-a9c9-560b74d31730 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:35.520559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef92cc9-37d5-4cff-a9c9-560b74d31730 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:35.531700 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d85b32-70dc-4ccf-b558-53f5574de1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:35.532234 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:35.533629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d85b32-70dc-4ccf-b558-53f5574de1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:35.534133 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d85b32-70dc-4ccf-b558-53f5574de1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:35.534643 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d85b32-70dc-4ccf-b558-53f5574de1a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:35.546242 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d85b32-70dc-4ccf-b558-53f5574de1a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:35.555285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e4d85b32-70dc-4ccf-b558-53f5574de1a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:35.555285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:35.555285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:35.555285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:35.555285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:35.555285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:35.555285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:35.555285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:35.555285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:35.555285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:35.555285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:35.555285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:35.555285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:35.555285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:35.555285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:35.555285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:35.556282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:35.556282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:35.556282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:35.556282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:35.556282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:35.556282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:35.556282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:35.556282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:35.556282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:35.556282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:35.556282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:35.556282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:35.556282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:35.556282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:35.556282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:35.556282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:35.557105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:35.557105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:35.557105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:35.557105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:35.557105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:35.557105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:35.557105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:35.557105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:35.557105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:35.557105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:35.557105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:35.557105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:35.559648 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 172/346] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:58:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:35.560635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d85b32-70dc-4ccf-b558-53f5574de1a9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:35.609868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:35.643283 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 175/347] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:35.645295 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7cfc86-ac30-4db1-8138-81525b7196e3 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:35.645589 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7cfc86-ac30-4db1-8138-81525b7196e3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:35.645933 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7cfc86-ac30-4db1-8138-81525b7196e3 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:35.651812 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 176/348] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:35.652632 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7cfc86-ac30-4db1-8138-81525b7196e3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:35.679061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d85b32-70dc-4ccf-b558-53f5574de1a9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:35.718133 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04fd60-7971-4ae4-83d7-e06ef38d456d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:35.719157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04fd60-7971-4ae4-83d7-e06ef38d456d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:35.719622 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04fd60-7971-4ae4-83d7-e06ef38d456d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:35.720076 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04fd60-7971-4ae4-83d7-e06ef38d456d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:35.721203 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f04fd60-7971-4ae4-83d7-e06ef38d456d None admin] RBAC: Adding query filter params (role.id=368ae945e96746faa1dc2ecbcefcf0f1, user.id=283f10fa85394c54bc993fab80930303, scope.system=all) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:35.721762 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f04fd60-7971-4ae4-83d7-e06ef38d456d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:35.724216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f04fd60-7971-4ae4-83d7-e06ef38d456d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:35.737705 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 173/349] 10.222.0.16 () {56 vars in 1308 bytes} [Fri Jun 24 03:58:35 2022] GET /identity/v3/role_assignments?user.id=283f10fa85394c54bc993fab80930303&scope.system=all&role.id=368ae945e96746faa1dc2ecbcefcf0f1 => generated 510 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:35.739707 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04fd60-7971-4ae4-83d7-e06ef38d456d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:35.778158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7cfc86-ac30-4db1-8138-81525b7196e3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:35.792347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:35.824893 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42671395-fc1c-4ad5-ba62-54e3d7cb93ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:35.827382 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42671395-fc1c-4ad5-ba62-54e3d7cb93ff None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:35.828592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42671395-fc1c-4ad5-ba62-54e3d7cb93ff None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:35.828592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42671395-fc1c-4ad5-ba62-54e3d7cb93ff None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:35.829398 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42671395-fc1c-4ad5-ba62-54e3d7cb93ff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:35.830180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42671395-fc1c-4ad5-ba62-54e3d7cb93ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:35.831802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42671395-fc1c-4ad5-ba62-54e3d7cb93ff None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:35.843546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 177/350] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:58:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:35.844940 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac184b-24f0-4a21-a966-e437ee3a014e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:35.845332 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac184b-24f0-4a21-a966-e437ee3a014e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:35.845709 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac184b-24f0-4a21-a966-e437ee3a014e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:35.867752 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:35.871372 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-77ac184b-24f0-4a21-a966-e437ee3a014e None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:35.872793 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-77ac184b-24f0-4a21-a966-e437ee3a014e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:35.876596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04fd60-7971-4ae4-83d7-e06ef38d456d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:35.920256 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81481caf-91c9-4cea-beea-87cf50becde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:35.921431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81481caf-91c9-4cea-beea-87cf50becde7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:35.921897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81481caf-91c9-4cea-beea-87cf50becde7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:35.922353 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81481caf-91c9-4cea-beea-87cf50becde7 None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:35.923855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81481caf-91c9-4cea-beea-87cf50becde7 None admin] RBAC: Authorizing `identity:get_project(project_id=40f51bfd8db04611b068f7edd7002004)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:35.926421 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81481caf-91c9-4cea-beea-87cf50becde7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:35.928735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 174/351] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:35 2022] GET /identity/v3/projects/40f51bfd8db04611b068f7edd7002004 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:35.929845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81481caf-91c9-4cea-beea-87cf50becde7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:35.963804 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 178/352] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:35.964765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac184b-24f0-4a21-a966-e437ee3a014e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:36.064075 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81481caf-91c9-4cea-beea-87cf50becde7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:36.121202 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df329fc-bc02-4c2b-92a3-579fa4ced2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:36.121785 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df329fc-bc02-4c2b-92a3-579fa4ced2bd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:36.122064 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df329fc-bc02-4c2b-92a3-579fa4ced2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:36.122340 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df329fc-bc02-4c2b-92a3-579fa4ced2bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:36.129931 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac184b-24f0-4a21-a966-e437ee3a014e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:36.131837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:36.160297 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:36.173555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34713895-818c-4b30-8425-d0a26a81bc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:36.174803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34713895-818c-4b30-8425-d0a26a81bc4d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:36.175349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34713895-818c-4b30-8425-d0a26a81bc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:36.176329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34713895-818c-4b30-8425-d0a26a81bc4d None admin] PATH_INFO: `/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:36.184603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34713895-818c-4b30-8425-d0a26a81bc4d None admin] RBAC: Authorizing `identity:get_group(group_id=95b3afd9130d4a7aad00b03a635a1f4f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:36.186437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34713895-818c-4b30-8425-d0a26a81bc4d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:36.188336 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 179/353] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jun 24 03:58:35 2022] GET /identity/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:36.189314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34713895-818c-4b30-8425-d0a26a81bc4d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:36.221171 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df329fc-bc02-4c2b-92a3-579fa4ced2bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:36.224587 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df329fc-bc02-4c2b-92a3-579fa4ced2bd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:36.336985 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34713895-818c-4b30-8425-d0a26a81bc4d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:36.373681 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 175/354] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:36.374935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df329fc-bc02-4c2b-92a3-579fa4ced2bd None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:36.379842 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5bd79-5185-441e-93b2-bac7fb4dcfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:36.380552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5bd79-5185-441e-93b2-bac7fb4dcfee None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:36.382166 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5bd79-5185-441e-93b2-bac7fb4dcfee None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:36.382166 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5bd79-5185-441e-93b2-bac7fb4dcfee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:36.399716 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c5bd79-5185-441e-93b2-bac7fb4dcfee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:36.408402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-e4c5bd79-5185-441e-93b2-bac7fb4dcfee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:36.408402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:36.408402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:36.408402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:36.408402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:36.408402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:36.408402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:36.408402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:36.408402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:36.408402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:36.408402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:36.408402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:36.408402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:36.408402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:36.408402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:36.408402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:36.409501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:36.409501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:36.409501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:36.409501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:36.409501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:36.409501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:36.409501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:36.409501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:36.409501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:36.409501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:36.409501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:36.409501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:36.409501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:36.409501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:36.409501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:36.409501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:36.410685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:36.410685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:36.410685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:36.410685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:36.410685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:36.410685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:36.410685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:36.410685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:36.410685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:36.410685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:36.410685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:36.410685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:36.410685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 180/355] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:36.411363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90739701-56fa-492a-8ba8-4a8e891572eb None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:36.411668 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90739701-56fa-492a-8ba8-4a8e891572eb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:36.411957 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90739701-56fa-492a-8ba8-4a8e891572eb None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:36.413384 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 181/356] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:36.417344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90739701-56fa-492a-8ba8-4a8e891572eb None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:36.447871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:36.449300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:36.503191 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df329fc-bc02-4c2b-92a3-579fa4ced2bd None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:36.542142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90739701-56fa-492a-8ba8-4a8e891572eb None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:36.546801 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735ef2b-8b0b-4e42-b774-f2e6d8f14333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:36.547475 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735ef2b-8b0b-4e42-b774-f2e6d8f14333 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:36.547759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735ef2b-8b0b-4e42-b774-f2e6d8f14333 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:36.548036 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735ef2b-8b0b-4e42-b774-f2e6d8f14333 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:36.548872 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735ef2b-8b0b-4e42-b774-f2e6d8f14333 None admin] RBAC: Adding query filter params (group.id=95b3afd9130d4a7aad00b03a635a1f4f, role.id=d7aadc20cd4c4838ae5d7d8e8826abae, scope.project.id=40f51bfd8db04611b068f7edd7002004) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:36.549191 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735ef2b-8b0b-4e42-b774-f2e6d8f14333 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:36.551246 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735ef2b-8b0b-4e42-b774-f2e6d8f14333 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:36.557744 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 176/357] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jun 24 03:58:36 2022] GET /identity/v3/role_assignments?group.id=95b3afd9130d4a7aad00b03a635a1f4f&scope.project.id=40f51bfd8db04611b068f7edd7002004&role.id=d7aadc20cd4c4838ae5d7d8e8826abae => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:36.559365 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524b116-e871-4534-9593-1a71f8713fac None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:36.559869 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524b116-e871-4534-9593-1a71f8713fac None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:36.560340 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524b116-e871-4534-9593-1a71f8713fac None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:36.561881 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 177/358] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:36.562664 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4524b116-e871-4534-9593-1a71f8713fac None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:36.602945 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ee1fe-bcfd-47c9-a4b3-da88ae484252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:36.603845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ee1fe-bcfd-47c9-a4b3-da88ae484252 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:36.604138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ee1fe-bcfd-47c9-a4b3-da88ae484252 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:36.604419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ee1fe-bcfd-47c9-a4b3-da88ae484252 None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:36.605761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466ee1fe-bcfd-47c9-a4b3-da88ae484252 None admin] RBAC: Authorizing `identity:get_project(project_id=40f51bfd8db04611b068f7edd7002004)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:36.607294 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466ee1fe-bcfd-47c9-a4b3-da88ae484252 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:36.609444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 182/359] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:36 2022] GET /identity/v3/projects/40f51bfd8db04611b068f7edd7002004 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:36.610690 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb04fe-0a1a-4468-ab78-c593302c5faf None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:36.610984 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb04fe-0a1a-4468-ab78-c593302c5faf None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:36.611264 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb04fe-0a1a-4468-ab78-c593302c5faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:36.631858 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c8bb04fe-0a1a-4468-ab78-c593302c5faf None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:36.633024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c8bb04fe-0a1a-4468-ab78-c593302c5faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:36.712412 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4524b116-e871-4534-9593-1a71f8713fac None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:36.736795 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 183/360] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:36.740788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c316d7-390b-4018-a96a-cfe9edc0fdcc None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:36.740993 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c316d7-390b-4018-a96a-cfe9edc0fdcc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:36.741071 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:36.741249 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c316d7-390b-4018-a96a-cfe9edc0fdcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:36.761135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-95c316d7-390b-4018-a96a-cfe9edc0fdcc None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:36.762923 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2236d-bba1-4879-b193-a9cfc1b9ea82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:36.763605 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2236d-bba1-4879-b193-a9cfc1b9ea82 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:36.763892 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2236d-bba1-4879-b193-a9cfc1b9ea82 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:36.764173 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2236d-bba1-4879-b193-a9cfc1b9ea82 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:36.764936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2236d-bba1-4879-b193-a9cfc1b9ea82 None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:36.765252 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2236d-bba1-4879-b193-a9cfc1b9ea82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:36.765884 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-95c316d7-390b-4018-a96a-cfe9edc0fdcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:36.766771 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2236d-bba1-4879-b193-a9cfc1b9ea82 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:36.778005 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 178/361] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:36.779096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2236d-bba1-4879-b193-a9cfc1b9ea82 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:36.787083 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:36.850468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 184/362] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:36.851678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c316d7-390b-4018-a96a-cfe9edc0fdcc None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:36.916763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2236d-bba1-4879-b193-a9cfc1b9ea82 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:36.955042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009f4c5-47aa-4ba2-a5c6-b1f5c0882bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:36.955945 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009f4c5-47aa-4ba2-a5c6-b1f5c0882bbb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:36.956235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009f4c5-47aa-4ba2-a5c6-b1f5c0882bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:36.956541 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009f4c5-47aa-4ba2-a5c6-b1f5c0882bbb None admin] PATH_INFO: `/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:36.957747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009f4c5-47aa-4ba2-a5c6-b1f5c0882bbb None admin] RBAC: Authorizing `identity:get_group(group_id=95b3afd9130d4a7aad00b03a635a1f4f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:36.959276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009f4c5-47aa-4ba2-a5c6-b1f5c0882bbb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:36.962047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 179/363] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jun 24 03:58:36 2022] GET /identity/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:36.962954 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009f4c5-47aa-4ba2-a5c6-b1f5c0882bbb None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:37.004961 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c316d7-390b-4018-a96a-cfe9edc0fdcc None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:37.032673 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:37.052183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f7b483-2119-4191-bbeb-93c952d0ed11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:37.053073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7b483-2119-4191-bbeb-93c952d0ed11 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:37.053368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7b483-2119-4191-bbeb-93c952d0ed11 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:37.053649 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7b483-2119-4191-bbeb-93c952d0ed11 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:37.065000 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f7b483-2119-4191-bbeb-93c952d0ed11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:37.074397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-99f7b483-2119-4191-bbeb-93c952d0ed11 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:37.074397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:37.074397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:37.074397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:37.074397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:37.074397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:37.074397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:37.074397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:37.074397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:37.074397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:37.074397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:37.074397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:37.074397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:37.074397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:37.074397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:37.074397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:37.075328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:37.075328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:37.075328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:37.075328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:37.075328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:37.075328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:37.075328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:37.075328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:37.075328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:37.075328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:37.075328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:37.075328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:37.075328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:37.075328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:37.075328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:37.075328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:37.076148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:37.076148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:37.076148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:37.076148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:37.076148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:37.076148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:37.076148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:37.076148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:37.076148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:37.076148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:37.076148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:37.076148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:37.076148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 185/364] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:58:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:37.076821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f7b483-2119-4191-bbeb-93c952d0ed11 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:37.101833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:37.107991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009f4c5-47aa-4ba2-a5c6-b1f5c0882bbb None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:37.156179 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548241da-9cbe-47f5-9e2f-2d2a9ab16b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:37.157571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548241da-9cbe-47f5-9e2f-2d2a9ab16b99 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:37.158735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548241da-9cbe-47f5-9e2f-2d2a9ab16b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:37.159025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548241da-9cbe-47f5-9e2f-2d2a9ab16b99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:37.212792 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f7b483-2119-4191-bbeb-93c952d0ed11 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:37.238441 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548241da-9cbe-47f5-9e2f-2d2a9ab16b99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:37.240414 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548241da-9cbe-47f5-9e2f-2d2a9ab16b99 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:37.256415 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8814489-8b96-4575-a775-6e150bb759cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:37.257225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8814489-8b96-4575-a775-6e150bb759cc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:37.257513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8814489-8b96-4575-a775-6e150bb759cc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:37.257789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8814489-8b96-4575-a775-6e150bb759cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:37.275606 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8814489-8b96-4575-a775-6e150bb759cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:37.284413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-e8814489-8b96-4575-a775-6e150bb759cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:58:37.284413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:37.284413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:37.284413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:37.284413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:37.284413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:37.284413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:37.284413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:37.284413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:37.284413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:37.284413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:37.284413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:37.284413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:37.284413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:37.284413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:37.284413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:37.285588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:37.285588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:37.285588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:37.285588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:37.285588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:37.285588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:37.285588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:37.285588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:37.285588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:37.285588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:37.285588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:37.285588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:37.285588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:37.285588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:37.285588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:37.285588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:37.286452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:37.286452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:37.286452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:37.286452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:37.286452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:37.286452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:37.286452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:37.286452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:37.286452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:37.286452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:37.286452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:58:37.286452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:37.287187 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 186/365] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:58:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:37.287992 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8814489-8b96-4575-a775-6e150bb759cc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:37.336480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:37.408023 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:37.422322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 180/366] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:37.424491 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548241da-9cbe-47f5-9e2f-2d2a9ab16b99 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:37.449728 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8814489-8b96-4575-a775-6e150bb759cc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:37.495395 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f9522-7f78-4ff9-82ee-39aeff191224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:37.495575 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f9522-7f78-4ff9-82ee-39aeff191224 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:37.495822 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f9522-7f78-4ff9-82ee-39aeff191224 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:37.496103 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f9522-7f78-4ff9-82ee-39aeff191224 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:37.496938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0f9522-7f78-4ff9-82ee-39aeff191224 None admin] RBAC: Adding query filter params (group.id=95b3afd9130d4a7aad00b03a635a1f4f, role.id=a048041c27f44edf98a110d23138a327, scope.project.id=40f51bfd8db04611b068f7edd7002004) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:37.497256 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0f9522-7f78-4ff9-82ee-39aeff191224 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:37.499533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0f9522-7f78-4ff9-82ee-39aeff191224 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:37.508645 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 187/367] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jun 24 03:58:37 2022] GET /identity/v3/role_assignments?group.id=95b3afd9130d4a7aad00b03a635a1f4f&scope.project.id=40f51bfd8db04611b068f7edd7002004&role.id=a048041c27f44edf98a110d23138a327 => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:37.509483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f9522-7f78-4ff9-82ee-39aeff191224 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:37.564883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548241da-9cbe-47f5-9e2f-2d2a9ab16b99 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:37.639849 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f9522-7f78-4ff9-82ee-39aeff191224 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:37.640443 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb2a24-ef4d-40d5-8fbf-f624994c2fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:37.641709 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb2a24-ef4d-40d5-8fbf-f624994c2fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:37.643126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb2a24-ef4d-40d5-8fbf-f624994c2fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:37.643126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb2a24-ef4d-40d5-8fbf-f624994c2fa4 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:37.643636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb2a24-ef4d-40d5-8fbf-f624994c2fa4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:37.643948 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb2a24-ef4d-40d5-8fbf-f624994c2fa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:37.647907 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb2a24-ef4d-40d5-8fbf-f624994c2fa4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:37.661991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 181/368] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:58:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:37.663688 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb2a24-ef4d-40d5-8fbf-f624994c2fa4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:37.686426 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ffdc7-b9e3-422b-88a7-3120a751f919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:37.687126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ffdc7-b9e3-422b-88a7-3120a751f919 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:37.687459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ffdc7-b9e3-422b-88a7-3120a751f919 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:37.687742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ffdc7-b9e3-422b-88a7-3120a751f919 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:37.688508 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ffdc7-b9e3-422b-88a7-3120a751f919 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:37.688817 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ffdc7-b9e3-422b-88a7-3120a751f919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:37.689907 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:37.690229 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ffdc7-b9e3-422b-88a7-3120a751f919 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:37.699010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:37.705217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 188/369] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:37.706049 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ffdc7-b9e3-422b-88a7-3120a751f919 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:37.812805 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb2a24-ef4d-40d5-8fbf-f624994c2fa4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:37.857998 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90574c4b-38ed-4faf-86b4-f30128ceb7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:37.859162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90574c4b-38ed-4faf-86b4-f30128ceb7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:37.859543 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90574c4b-38ed-4faf-86b4-f30128ceb7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:37.859826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90574c4b-38ed-4faf-86b4-f30128ceb7e8 None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:37.861471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90574c4b-38ed-4faf-86b4-f30128ceb7e8 None admin] RBAC: Authorizing `identity:get_project(project_id=e87c7eebde3941f1aa5f40ce794e8687)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:37.863241 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90574c4b-38ed-4faf-86b4-f30128ceb7e8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:37.865368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 182/370] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:37 2022] GET /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:37.866363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90574c4b-38ed-4faf-86b4-f30128ceb7e8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:37.872271 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ffdc7-b9e3-422b-88a7-3120a751f919 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:37.937037 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc944332-1557-4490-9d06-e3c5f929d28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:37.938944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc944332-1557-4490-9d06-e3c5f929d28f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:37.939456 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc944332-1557-4490-9d06-e3c5f929d28f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:37.939936 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc944332-1557-4490-9d06-e3c5f929d28f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:38.005520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:38.028308 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:38.033507 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc944332-1557-4490-9d06-e3c5f929d28f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:38.033507 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc944332-1557-4490-9d06-e3c5f929d28f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:38.035531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90574c4b-38ed-4faf-86b4-f30128ceb7e8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:38.090471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ff5cc-90b9-4650-a7b1-58336d69fd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:38.090471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ff5cc-90b9-4650-a7b1-58336d69fd1f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:38.090471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ff5cc-90b9-4650-a7b1-58336d69fd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:38.090471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ff5cc-90b9-4650-a7b1-58336d69fd1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:38.180262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 189/371] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:38.181298 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc944332-1557-4490-9d06-e3c5f929d28f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:38.193935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ff5cc-90b9-4650-a7b1-58336d69fd1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:38.195607 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ff5cc-90b9-4650-a7b1-58336d69fd1f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:38.291786 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:38.292390 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:38.295397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc944332-1557-4490-9d06-e3c5f929d28f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:38.312488 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 183/372] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:38.313414 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ff5cc-90b9-4650-a7b1-58336d69fd1f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:38.334010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7acb71-3373-4c0d-8d64-b1fcd63e16bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:38.334922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7acb71-3373-4c0d-8d64-b1fcd63e16bc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:38.335208 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7acb71-3373-4c0d-8d64-b1fcd63e16bc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:38.335488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7acb71-3373-4c0d-8d64-b1fcd63e16bc None admin] PATH_INFO: `/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:38.336738 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7acb71-3373-4c0d-8d64-b1fcd63e16bc None admin] RBAC: Authorizing `identity:get_group(group_id=95b3afd9130d4a7aad00b03a635a1f4f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:38.340922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7acb71-3373-4c0d-8d64-b1fcd63e16bc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:38.340922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 190/373] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jun 24 03:58:38 2022] GET /identity/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:38.340922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7acb71-3373-4c0d-8d64-b1fcd63e16bc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:38.426096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ff5cc-90b9-4650-a7b1-58336d69fd1f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:38.461327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7acb71-3373-4c0d-8d64-b1fcd63e16bc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:38.466502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f22b8-cb3f-4684-937d-0b93c07e0e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:38.467603 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f22b8-cb3f-4684-937d-0b93c07e0e44 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:38.467893 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f22b8-cb3f-4684-937d-0b93c07e0e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:38.468247 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f22b8-cb3f-4684-937d-0b93c07e0e44 None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:38.469711 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f22b8-cb3f-4684-937d-0b93c07e0e44 None admin] RBAC: Authorizing `identity:get_project(project_id=e87c7eebde3941f1aa5f40ce794e8687)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:38.471303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259f22b8-cb3f-4684-937d-0b93c07e0e44 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:38.473726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 184/374] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:38 2022] GET /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:38.480217 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f22b8-cb3f-4684-937d-0b93c07e0e44 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:38.515245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ad4aa-b54c-4ef3-9bd1-0acf73e9ac85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:38.516776 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ad4aa-b54c-4ef3-9bd1-0acf73e9ac85 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:38.517288 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ad4aa-b54c-4ef3-9bd1-0acf73e9ac85 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:38.517791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ad4aa-b54c-4ef3-9bd1-0acf73e9ac85 None admin] PATH_INFO: `/v3/projects/6a1c0f0b20b844d7b8311813151d93aa` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:38.520789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ad4aa-b54c-4ef3-9bd1-0acf73e9ac85 None admin] RBAC: Authorizing `identity:get_project(project_id=6a1c0f0b20b844d7b8311813151d93aa)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:38.522970 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ad4aa-b54c-4ef3-9bd1-0acf73e9ac85 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:38.525132 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 191/375] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:38 2022] GET /identity/v3/projects/6a1c0f0b20b844d7b8311813151d93aa => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:38.526535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ad4aa-b54c-4ef3-9bd1-0acf73e9ac85 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:38.540799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:38.570911 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:38.655925 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f22b8-cb3f-4684-937d-0b93c07e0e44 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:38.661350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ad4aa-b54c-4ef3-9bd1-0acf73e9ac85 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:38.710249 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b01d5-d4ad-418f-831f-afdb89ca1e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:38.711133 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b01d5-d4ad-418f-831f-afdb89ca1e68 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:38.711420 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b01d5-d4ad-418f-831f-afdb89ca1e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:38.711675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a23065-d790-42e2-8500-3b6b85c751bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:38.711787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b01d5-d4ad-418f-831f-afdb89ca1e68 None admin] PATH_INFO: `/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:38.712632 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a23065-d790-42e2-8500-3b6b85c751bc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:38.712907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b01d5-d4ad-418f-831f-afdb89ca1e68 None admin] RBAC: Authorizing `identity:get_group(group_id=95b3afd9130d4a7aad00b03a635a1f4f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:38.713314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a23065-d790-42e2-8500-3b6b85c751bc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:38.713774 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a23065-d790-42e2-8500-3b6b85c751bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:38.714361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b01d5-d4ad-418f-831f-afdb89ca1e68 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:38.715040 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a23065-d790-42e2-8500-3b6b85c751bc None admin] RBAC: Adding query filter params (group.id=95b3afd9130d4a7aad00b03a635a1f4f, role.id=d7aadc20cd4c4838ae5d7d8e8826abae, scope.project.id=e87c7eebde3941f1aa5f40ce794e8687) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:38.715542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a23065-d790-42e2-8500-3b6b85c751bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:38.716027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 192/376] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jun 24 03:58:38 2022] GET /identity/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:38.716882 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b01d5-d4ad-418f-831f-afdb89ca1e68 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:38.717669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a23065-d790-42e2-8500-3b6b85c751bc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:38.725156 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 185/377] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jun 24 03:58:38 2022] GET /identity/v3/role_assignments?group.id=95b3afd9130d4a7aad00b03a635a1f4f&scope.project.id=e87c7eebde3941f1aa5f40ce794e8687&role.id=d7aadc20cd4c4838ae5d7d8e8826abae => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:38.726391 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a23065-d790-42e2-8500-3b6b85c751bc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:38.875525 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b01d5-d4ad-418f-831f-afdb89ca1e68 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:38.880326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:38.889012 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a23065-d790-42e2-8500-3b6b85c751bc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:38.910575 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:38.927945 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf329c7-bcb4-492e-aa6d-90d9f7e21a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:38.929344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf329c7-bcb4-492e-aa6d-90d9f7e21a80 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:38.929701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf329c7-bcb4-492e-aa6d-90d9f7e21a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:38.929988 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf329c7-bcb4-492e-aa6d-90d9f7e21a80 None admin] PATH_INFO: `/v3/groups/445579294e72451cbcc55238bc259ced` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:38.938487 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf329c7-bcb4-492e-aa6d-90d9f7e21a80 None admin] RBAC: Authorizing `identity:get_group(group_id=445579294e72451cbcc55238bc259ced)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:38.938487 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf329c7-bcb4-492e-aa6d-90d9f7e21a80 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:38.939905 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8147dac-dd5f-4d3f-8747-8d40d856d969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:38.940257 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 193/378] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jun 24 03:58:38 2022] GET /identity/v3/groups/445579294e72451cbcc55238bc259ced => generated 229 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:38.940692 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8147dac-dd5f-4d3f-8747-8d40d856d969 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:38.941545 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8147dac-dd5f-4d3f-8747-8d40d856d969 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:38.942195 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8147dac-dd5f-4d3f-8747-8d40d856d969 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:38.942357 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1d464-6f8e-4d09-aba8-7eab7b673594 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:38.945792 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1d464-6f8e-4d09-aba8-7eab7b673594 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:38.945910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8147dac-dd5f-4d3f-8747-8d40d856d969 None admin] RBAC: Adding query filter params (group.id=95b3afd9130d4a7aad00b03a635a1f4f, role.id=a048041c27f44edf98a110d23138a327, scope.project.id=e87c7eebde3941f1aa5f40ce794e8687) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:38.945985 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1d464-6f8e-4d09-aba8-7eab7b673594 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:38.946071 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8147dac-dd5f-4d3f-8747-8d40d856d969 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:38.946140 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 194/379] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:38.946204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8147dac-dd5f-4d3f-8747-8d40d856d969 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:38.947355 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1d464-6f8e-4d09-aba8-7eab7b673594 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:38.954003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 186/380] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jun 24 03:58:38 2022] GET /identity/v3/role_assignments?group.id=95b3afd9130d4a7aad00b03a635a1f4f&scope.project.id=e87c7eebde3941f1aa5f40ce794e8687&role.id=a048041c27f44edf98a110d23138a327 => generated 260 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:38.955439 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf2fb7-8882-4b32-a971-4545d4b98484 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:38.956102 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf2fb7-8882-4b32-a971-4545d4b98484 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:38.957578 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf2fb7-8882-4b32-a971-4545d4b98484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:38.976682 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-79cf2fb7-8882-4b32-a971-4545d4b98484 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:38.978243 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-79cf2fb7-8882-4b32-a971-4545d4b98484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:39.130527 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 187/381] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:39.176181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf2fb7-8882-4b32-a971-4545d4b98484 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:39.189859 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1d464-6f8e-4d09-aba8-7eab7b673594 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:39.233209 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7866a5-3608-4c72-86f6-190d4dd2025d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:39.233899 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7866a5-3608-4c72-86f6-190d4dd2025d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:39.234185 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7866a5-3608-4c72-86f6-190d4dd2025d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:39.234501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7866a5-3608-4c72-86f6-190d4dd2025d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:39.235512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7866a5-3608-4c72-86f6-190d4dd2025d None admin] RBAC: Adding query filter params (group.id=445579294e72451cbcc55238bc259ced, role.id=f0f7e28cbe314921afdfb657b163436f, scope.project.id=6a1c0f0b20b844d7b8311813151d93aa) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:39.235827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7866a5-3608-4c72-86f6-190d4dd2025d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:39.237607 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7866a5-3608-4c72-86f6-190d4dd2025d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:39.245208 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 195/382] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jun 24 03:58:38 2022] GET /identity/v3/role_assignments?group.id=445579294e72451cbcc55238bc259ced&scope.project.id=6a1c0f0b20b844d7b8311813151d93aa&role.id=f0f7e28cbe314921afdfb657b163436f => generated 260 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:39.369010 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cf2fb7-8882-4b32-a971-4545d4b98484 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:39.381107 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:39.410079 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe4101-c528-44c5-b24d-f6cfb0f5562e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:39.411788 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe4101-c528-44c5-b24d-f6cfb0f5562e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:39.412444 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe4101-c528-44c5-b24d-f6cfb0f5562e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:39.413079 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe4101-c528-44c5-b24d-f6cfb0f5562e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:39.432042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe4101-c528-44c5-b24d-f6cfb0f5562e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:39.445755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-a6fe4101-c528-44c5-b24d-f6cfb0f5562e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:39.445755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:39.445755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:39.445755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:39.445755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:39.445755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:39.445755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:39.445755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:39.445755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:39.445755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:39.445755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:39.445755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:39.445755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:39.445755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:39.445755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:39.445755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:39.447260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:39.447260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:39.447260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:39.447260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:39.447260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:39.447260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:39.447260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:39.447260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:39.447260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:39.447260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:39.447260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:39.447260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:39.447260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:39.447260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:39.447260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:39.447260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:39.448708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:39.448708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:39.448708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:39.448708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:39.448708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:39.448708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:39.448708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:39.448708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:39.448708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:39.448708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:39.448708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:39.448708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:39.451774 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 188/383] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:58:39.457972 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7866a5-3608-4c72-86f6-190d4dd2025d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:39.517093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:39.619239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7866a5-3608-4c72-86f6-190d4dd2025d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:39.660176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27543d53-a2b4-44a4-bd20-ace5d4ddf04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:39.661163 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27543d53-a2b4-44a4-bd20-ace5d4ddf04a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:39.661592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27543d53-a2b4-44a4-bd20-ace5d4ddf04a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:39.662008 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27543d53-a2b4-44a4-bd20-ace5d4ddf04a None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:39.662941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27543d53-a2b4-44a4-bd20-ace5d4ddf04a None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:39.663393 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27543d53-a2b4-44a4-bd20-ace5d4ddf04a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:39.664998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27543d53-a2b4-44a4-bd20-ace5d4ddf04a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:39.676094 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 196/384] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:39.681710 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe4101-c528-44c5-b24d-f6cfb0f5562e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:39.791992 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe4101-c528-44c5-b24d-f6cfb0f5562e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:39.831334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56ccec-2719-463d-bda7-0be64804fb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:39.833090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56ccec-2719-463d-bda7-0be64804fb42 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:39.833521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56ccec-2719-463d-bda7-0be64804fb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:39.833940 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56ccec-2719-463d-bda7-0be64804fb42 None admin] PATH_INFO: `/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:39.835334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56ccec-2719-463d-bda7-0be64804fb42 None admin] RBAC: Authorizing `identity:get_group(group_id=95b3afd9130d4a7aad00b03a635a1f4f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:39.836962 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56ccec-2719-463d-bda7-0be64804fb42 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:39.839135 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 189/385] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jun 24 03:58:39 2022] GET /identity/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:39.844344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27543d53-a2b4-44a4-bd20-ace5d4ddf04a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:39.987284 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:39.997544 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27543d53-a2b4-44a4-bd20-ace5d4ddf04a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:40.049937 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae6d74-fe9b-4268-a5c4-c864c580b9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:40.050818 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae6d74-fe9b-4268-a5c4-c864c580b9bd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:40.051106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae6d74-fe9b-4268-a5c4-c864c580b9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:40.051388 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae6d74-fe9b-4268-a5c4-c864c580b9bd None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:40.055106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae6d74-fe9b-4268-a5c4-c864c580b9bd None admin] RBAC: Authorizing `identity:get_project(project_id=40f51bfd8db04611b068f7edd7002004)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:40.056572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae6d74-fe9b-4268-a5c4-c864c580b9bd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:40.058674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 197/386] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:39 2022] GET /identity/v3/projects/40f51bfd8db04611b068f7edd7002004 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:40.063995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56ccec-2719-463d-bda7-0be64804fb42 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:40.119619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:40.191347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56ccec-2719-463d-bda7-0be64804fb42 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:40.192309 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982aaf9-7bcd-40e5-bae7-f352771504ce None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:40.192654 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982aaf9-7bcd-40e5-bae7-f352771504ce None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:40.193006 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e982aaf9-7bcd-40e5-bae7-f352771504ce None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:40.194752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 198/387] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:40.199543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615675d-d7f7-4688-b27f-c188d1992f3d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:40.199899 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615675d-d7f7-4688-b27f-c188d1992f3d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:40.200272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615675d-d7f7-4688-b27f-c188d1992f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:40.230940 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3bf0f-e25a-449d-b912-983c2f5d32c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:40.231967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3bf0f-e25a-449d-b912-983c2f5d32c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:40.232253 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3bf0f-e25a-449d-b912-983c2f5d32c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:40.232534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3bf0f-e25a-449d-b912-983c2f5d32c4 None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004/groups/95b3afd9130d4a7aad00b03a635a1f4f/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:40.234245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d615675d-d7f7-4688-b27f-c188d1992f3d None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:40.234716 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3bf0f-e25a-449d-b912-983c2f5d32c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=40f51bfd8db04611b068f7edd7002004, group_id=95b3afd9130d4a7aad00b03a635a1f4f, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:40.235490 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d615675d-d7f7-4688-b27f-c188d1992f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:40.236161 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3bf0f-e25a-449d-b912-983c2f5d32c4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:40.310978 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 190/388] 10.222.0.16 () {58 vars in 1442 bytes} [Fri Jun 24 03:58:40 2022] PUT /identity/v3/projects/40f51bfd8db04611b068f7edd7002004/groups/95b3afd9130d4a7aad00b03a635a1f4f/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:58:40.331704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 199/389] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:40.377548 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3bf0f-e25a-449d-b912-983c2f5d32c4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:40.502766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:40.507229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3bf0f-e25a-449d-b912-983c2f5d32c4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:40.547925 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c4654-8ecd-4cf6-a9a4-ebcbd201872e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:40.548744 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c4654-8ecd-4cf6-a9a4-ebcbd201872e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:40.549035 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c4654-8ecd-4cf6-a9a4-ebcbd201872e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:40.549319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c4654-8ecd-4cf6-a9a4-ebcbd201872e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:40.562682 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559c4654-8ecd-4cf6-a9a4-ebcbd201872e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:40.572115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-559c4654-8ecd-4cf6-a9a4-ebcbd201872e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:40.572115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:40.572115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:40.572115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:40.572115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:40.572115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:40.572115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:40.572115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:40.572115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:40.572115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:40.572115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:40.572115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:40.572115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:40.572115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:40.572115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:40.572115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:40.573030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:40.573030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:40.573030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:40.573030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:40.573030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:40.573030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:40.573030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:40.573030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:40.573030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:40.573030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:40.573030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:40.573030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:40.573030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:40.573030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:40.573030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:40.573030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:40.574524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:40.574524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:40.574524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:40.574524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:40.574524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:40.574524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:40.574524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:40.574524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:40.574524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:40.574524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:40.574524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:40.574524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:40.574524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 191/390] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:58:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:40.579531 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615675d-d7f7-4688-b27f-c188d1992f3d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:40.702670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:40.714640 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615675d-d7f7-4688-b27f-c188d1992f3d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:40.752043 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a8055-2d07-48a2-9a14-b96fdaa86680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:40.752723 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a8055-2d07-48a2-9a14-b96fdaa86680 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:40.753008 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a8055-2d07-48a2-9a14-b96fdaa86680 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:40.753337 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a8055-2d07-48a2-9a14-b96fdaa86680 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:40.754201 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a8055-2d07-48a2-9a14-b96fdaa86680 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:40.754543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a8055-2d07-48a2-9a14-b96fdaa86680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:40.755979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a8055-2d07-48a2-9a14-b96fdaa86680 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:40.773811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 200/391] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:58:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:40.778926 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c4654-8ecd-4cf6-a9a4-ebcbd201872e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:40.918926 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559c4654-8ecd-4cf6-a9a4-ebcbd201872e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:40.959002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df4b3a-c910-41b2-9465-f55750ea2667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:40.959882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df4b3a-c910-41b2-9465-f55750ea2667 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:40.960167 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df4b3a-c910-41b2-9465-f55750ea2667 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:40.960447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df4b3a-c910-41b2-9465-f55750ea2667 None admin] PATH_INFO: `/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:40.961684 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df4b3a-c910-41b2-9465-f55750ea2667 None admin] RBAC: Authorizing `identity:get_group(group_id=95b3afd9130d4a7aad00b03a635a1f4f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:40.963175 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76df4b3a-c910-41b2-9465-f55750ea2667 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:40.965080 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 192/392] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jun 24 03:58:40 2022] GET /identity/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:40.971657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a8055-2d07-48a2-9a14-b96fdaa86680 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:41.110483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a8055-2d07-48a2-9a14-b96fdaa86680 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:41.157763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b6831-56e0-4e07-96ae-5832668a5853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:41.158685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b6831-56e0-4e07-96ae-5832668a5853 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:41.159138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b6831-56e0-4e07-96ae-5832668a5853 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:41.159627 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b6831-56e0-4e07-96ae-5832668a5853 None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:41.167497 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b6831-56e0-4e07-96ae-5832668a5853 None admin] RBAC: Authorizing `identity:get_project(project_id=40f51bfd8db04611b068f7edd7002004)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:41.169211 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b6831-56e0-4e07-96ae-5832668a5853 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:41.171357 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 201/393] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:40 2022] GET /identity/v3/projects/40f51bfd8db04611b068f7edd7002004 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:41.178033 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df4b3a-c910-41b2-9465-f55750ea2667 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:41.225361 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:41.327228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df4b3a-c910-41b2-9465-f55750ea2667 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:41.383027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac0aac-abd1-466b-815c-5293ad6b2cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:41.384360 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac0aac-abd1-466b-815c-5293ad6b2cfa None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:41.384764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac0aac-abd1-466b-815c-5293ad6b2cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:41.385158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac0aac-abd1-466b-815c-5293ad6b2cfa None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004/groups/95b3afd9130d4a7aad00b03a635a1f4f/roles/a048041c27f44edf98a110d23138a327` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:41.393073 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac0aac-abd1-466b-815c-5293ad6b2cfa None admin] RBAC: Authorizing `identity:create_grant(project_id=40f51bfd8db04611b068f7edd7002004, group_id=95b3afd9130d4a7aad00b03a635a1f4f, role_id=a048041c27f44edf98a110d23138a327)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:41.395996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac0aac-abd1-466b-815c-5293ad6b2cfa None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:41.479381 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 193/394] 10.222.0.16 () {58 vars in 1442 bytes} [Fri Jun 24 03:58:41 2022] PUT /identity/v3/projects/40f51bfd8db04611b068f7edd7002004/groups/95b3afd9130d4a7aad00b03a635a1f4f/roles/a048041c27f44edf98a110d23138a327 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:58:41.838962 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c435b38-c2e8-4f81-a23a-eabb0533c957 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:41.839960 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c435b38-c2e8-4f81-a23a-eabb0533c957 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:41.840442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c435b38-c2e8-4f81-a23a-eabb0533c957 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:41.841654 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7d656-f1c6-4250-91ed-6b3eea368649 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:41.842116 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 202/395] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:41.842568 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7d656-f1c6-4250-91ed-6b3eea368649 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:41.842999 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df7d656-f1c6-4250-91ed-6b3eea368649 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:41.844572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 194/396] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:41.847568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f498f-f068-4fdf-bdd6-de7662e9ba19 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:41.847864 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f498f-f068-4fdf-bdd6-de7662e9ba19 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:41.848146 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f498f-f068-4fdf-bdd6-de7662e9ba19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:41.850996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e92088-54d4-4c8d-96db-12b2a137254a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:41.851590 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e92088-54d4-4c8d-96db-12b2a137254a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:41.852065 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e92088-54d4-4c8d-96db-12b2a137254a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:41.870285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-dc2f498f-f068-4fdf-bdd6-de7662e9ba19 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:41.871572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dc2f498f-f068-4fdf-bdd6-de7662e9ba19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:41.871720 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-63e92088-54d4-4c8d-96db-12b2a137254a None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:41.872919 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-63e92088-54d4-4c8d-96db-12b2a137254a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:41.903552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:41.971529 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 195/397] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:58:41.974168 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89001f83-a7df-4d1e-82b4-d3d007481b3a None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:41.974626 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89001f83-a7df-4d1e-82b4-d3d007481b3a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:41.975026 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89001f83-a7df-4d1e-82b4-d3d007481b3a None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:41.980680 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 196/398] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:41.983403 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68586ae8-9035-4cde-97ac-0c75815faee0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:41.983869 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68586ae8-9035-4cde-97ac-0c75815faee0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:41.984406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68586ae8-9035-4cde-97ac-0c75815faee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:42.001642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 203/399] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:42.006215 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-68586ae8-9035-4cde-97ac-0c75815faee0 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:42.007651 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-68586ae8-9035-4cde-97ac-0c75815faee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:42.012212 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f498f-f068-4fdf-bdd6-de7662e9ba19 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:42.021595 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:42.114581 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 197/400] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:58:42.116657 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68586ae8-9035-4cde-97ac-0c75815faee0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:42.165258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f498f-f068-4fdf-bdd6-de7662e9ba19 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:42.203854 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e3349-bba0-4cc4-a1dd-c8713e474e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:42.204702 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e3349-bba0-4cc4-a1dd-c8713e474e2f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:42.204989 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e3349-bba0-4cc4-a1dd-c8713e474e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:42.205295 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e3349-bba0-4cc4-a1dd-c8713e474e2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:42.220780 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e3349-bba0-4cc4-a1dd-c8713e474e2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:42.231802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-355e3349-bba0-4cc4-a1dd-c8713e474e2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:42.231802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:42.231802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:42.231802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:42.231802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:42.231802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:42.231802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:42.231802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:42.231802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:42.231802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:42.231802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:42.231802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:42.231802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:42.231802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:42.231802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:42.231802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:42.232708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:42.232708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:42.232708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:42.232708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:42.232708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:42.232708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:42.232708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:42.232708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:42.232708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:42.232708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:42.232708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:42.232708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:42.232708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:42.232708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:42.232708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:42.232708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:42.233515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:42.233515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:42.233515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:42.233515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:42.233515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:42.233515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:42.233515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:42.233515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:42.233515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:42.233515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:42.233515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:42.233515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:42.235888 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68586ae8-9035-4cde-97ac-0c75815faee0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:42.238271 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 204/401] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:58:42.240304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e3349-bba0-4cc4-a1dd-c8713e474e2f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:42.263096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:42.276647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd17875-b45c-47d1-9de3-e10b26df424c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:42.279496 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd17875-b45c-47d1-9de3-e10b26df424c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:42.279496 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd17875-b45c-47d1-9de3-e10b26df424c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:42.279496 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd17875-b45c-47d1-9de3-e10b26df424c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:42.290558 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd17875-b45c-47d1-9de3-e10b26df424c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:42.299274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-edd17875-b45c-47d1-9de3-e10b26df424c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:42.299274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:42.299274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:42.299274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:42.299274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:42.299274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:42.299274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:42.299274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:42.299274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:42.299274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:42.299274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:42.299274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:42.299274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:42.299274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:42.299274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:42.299274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:42.300212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:42.300212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:42.300212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:42.300212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:42.300212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:42.300212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:42.300212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:42.300212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:42.300212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:42.300212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:42.300212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:42.300212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:42.300212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:42.300212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:42.300212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:42.300212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:42.301128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:42.301128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:42.301128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:42.301128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:42.301128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:42.301128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:42.301128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:42.301128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:42.301128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:42.301128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:42.301128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:42.301128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:42.303707 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 198/402] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:58:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:42.304707 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd17875-b45c-47d1-9de3-e10b26df424c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:42.310673 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:42.380546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e3349-bba0-4cc4-a1dd-c8713e474e2f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:42.426856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673bd6d-4ea1-4d18-af43-89bb23bc30b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:42.427298 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd17875-b45c-47d1-9de3-e10b26df424c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:42.428149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673bd6d-4ea1-4d18-af43-89bb23bc30b9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:42.428638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673bd6d-4ea1-4d18-af43-89bb23bc30b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:42.429135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2673bd6d-4ea1-4d18-af43-89bb23bc30b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:42.440097 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2673bd6d-4ea1-4d18-af43-89bb23bc30b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:42.448848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-2673bd6d-4ea1-4d18-af43-89bb23bc30b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:58:42.448848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:42.448848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:42.448848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:42.448848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:42.448848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:42.448848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:42.448848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:42.448848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:42.448848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:42.448848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:42.448848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:42.448848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:42.448848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:42.448848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:42.448848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:42.449873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:42.449873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:42.449873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:42.449873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:42.449873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:42.449873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:42.449873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:42.449873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:42.449873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:42.449873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:42.449873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:42.449873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:42.449873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:42.449873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:42.449873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:42.449873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:42.450693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:42.450693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:42.450693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:42.450693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:42.450693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:42.450693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:42.450693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:42.450693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:42.450693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:42.450693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:42.450693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:58:42.450693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:42.453421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 205/403] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:58:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:42.454571 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673bd6d-4ea1-4d18-af43-89bb23bc30b9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:42.470815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed3780-f91d-46a5-a666-ac06619871bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:42.471501 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed3780-f91d-46a5-a666-ac06619871bb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:42.471789 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed3780-f91d-46a5-a666-ac06619871bb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:42.472643 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed3780-f91d-46a5-a666-ac06619871bb None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:42.473416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed3780-f91d-46a5-a666-ac06619871bb None admin] RBAC: Adding query filter params (name=member) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:42.473727 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed3780-f91d-46a5-a666-ac06619871bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:42.476049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed3780-f91d-46a5-a666-ac06619871bb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:42.490453 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 199/404] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:42.491676 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed3780-f91d-46a5-a666-ac06619871bb None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:42.547796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:42.568042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:42.580000 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2673bd6d-4ea1-4d18-af43-89bb23bc30b9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:42.608867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed3780-f91d-46a5-a666-ac06619871bb None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:42.617835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895622bc-2710-4ee3-b065-7217a35fda83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:42.618549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895622bc-2710-4ee3-b065-7217a35fda83 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:42.619100 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895622bc-2710-4ee3-b065-7217a35fda83 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:42.619387 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895622bc-2710-4ee3-b065-7217a35fda83 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:42.620197 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895622bc-2710-4ee3-b065-7217a35fda83 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:42.620816 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895622bc-2710-4ee3-b065-7217a35fda83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:42.621899 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895622bc-2710-4ee3-b065-7217a35fda83 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:42.634709 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 206/405] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:58:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:42.635563 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895622bc-2710-4ee3-b065-7217a35fda83 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:42.649326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbeb524-a352-4750-8e61-06800a4db01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:42.650325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbeb524-a352-4750-8e61-06800a4db01e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:42.650801 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbeb524-a352-4750-8e61-06800a4db01e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:42.651224 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbeb524-a352-4750-8e61-06800a4db01e None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:42.652115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbeb524-a352-4750-8e61-06800a4db01e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:42.652572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbeb524-a352-4750-8e61-06800a4db01e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:42.654164 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbeb524-a352-4750-8e61-06800a4db01e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:42.663903 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 200/406] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:42.664996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbeb524-a352-4750-8e61-06800a4db01e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:42.741844 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895622bc-2710-4ee3-b065-7217a35fda83 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:42.773219 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbeb524-a352-4750-8e61-06800a4db01e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:42.778712 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92574e17-6073-4465-9949-8d7cb00fe500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:42.779586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92574e17-6073-4465-9949-8d7cb00fe500 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:42.779872 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92574e17-6073-4465-9949-8d7cb00fe500 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:42.780152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92574e17-6073-4465-9949-8d7cb00fe500 None admin] PATH_INFO: `/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:42.781358 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92574e17-6073-4465-9949-8d7cb00fe500 None admin] RBAC: Authorizing `identity:get_group(group_id=95b3afd9130d4a7aad00b03a635a1f4f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:42.782867 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92574e17-6073-4465-9949-8d7cb00fe500 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:42.784546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 207/407] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jun 24 03:58:42 2022] GET /identity/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:42.790176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92574e17-6073-4465-9949-8d7cb00fe500 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:42.792975 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:42.804964 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:42.813764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41ab58-8585-4bee-b1c5-8e08f7162918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:42.814989 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41ab58-8585-4bee-b1c5-8e08f7162918 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:42.818083 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41ab58-8585-4bee-b1c5-8e08f7162918 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:42.818083 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41ab58-8585-4bee-b1c5-8e08f7162918 None admin] PATH_INFO: `/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:42.818083 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba41ab58-8585-4bee-b1c5-8e08f7162918 None admin] RBAC: Authorizing `identity:get_group(group_id=95b3afd9130d4a7aad00b03a635a1f4f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:42.819229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba41ab58-8585-4bee-b1c5-8e08f7162918 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:42.821399 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 201/408] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jun 24 03:58:42 2022] GET /identity/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:42.822456 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41ab58-8585-4bee-b1c5-8e08f7162918 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:42.902792 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92574e17-6073-4465-9949-8d7cb00fe500 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:42.934499 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41ab58-8585-4bee-b1c5-8e08f7162918 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:42.941986 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b59fa-f549-4d9d-9f99-4b43613a8963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:42.943311 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b59fa-f549-4d9d-9f99-4b43613a8963 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:42.943610 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b59fa-f549-4d9d-9f99-4b43613a8963 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:42.943894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b59fa-f549-4d9d-9f99-4b43613a8963 None admin] PATH_INFO: `/v3/groups/445579294e72451cbcc55238bc259ced` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:42.945106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b59fa-f549-4d9d-9f99-4b43613a8963 None admin] RBAC: Authorizing `identity:get_group(group_id=445579294e72451cbcc55238bc259ced)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:42.946945 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b59fa-f549-4d9d-9f99-4b43613a8963 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:42.948895 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 208/409] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jun 24 03:58:42 2022] GET /identity/v3/groups/445579294e72451cbcc55238bc259ced => generated 229 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:42.949658 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b59fa-f549-4d9d-9f99-4b43613a8963 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:42.973430 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bc6677-84ef-48e8-be58-619af269d0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:42.974410 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc6677-84ef-48e8-be58-619af269d0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:42.974768 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc6677-84ef-48e8-be58-619af269d0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:42.975411 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bc6677-84ef-48e8-be58-619af269d0a4 None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:42.976327 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc6677-84ef-48e8-be58-619af269d0a4 None admin] RBAC: Authorizing `identity:get_project(project_id=e87c7eebde3941f1aa5f40ce794e8687)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:42.977796 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bc6677-84ef-48e8-be58-619af269d0a4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:42.979699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 202/410] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:42 2022] GET /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:42.985231 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714fe30-c446-4f05-b96b-2b341808b1b7 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:42.985522 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714fe30-c446-4f05-b96b-2b341808b1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:42.985804 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8714fe30-c446-4f05-b96b-2b341808b1b7 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:42.987222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 203/411] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:42.988375 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714fe30-c446-4f05-b96b-2b341808b1b7 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:43.029936 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:43.059466 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:43.068618 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b59fa-f549-4d9d-9f99-4b43613a8963 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:43.099320 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8714fe30-c446-4f05-b96b-2b341808b1b7 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:43.105408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199e898-5f2a-4d3b-be06-f126a5be2016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:43.106287 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199e898-5f2a-4d3b-be06-f126a5be2016 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:43.106606 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199e898-5f2a-4d3b-be06-f126a5be2016 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:43.106893 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199e898-5f2a-4d3b-be06-f126a5be2016 None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:43.108233 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199e898-5f2a-4d3b-be06-f126a5be2016 None admin] RBAC: Authorizing `identity:get_project(project_id=e87c7eebde3941f1aa5f40ce794e8687)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:43.109939 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199e898-5f2a-4d3b-be06-f126a5be2016 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:43.117581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 209/412] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:42 2022] GET /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:43.117581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199e898-5f2a-4d3b-be06-f126a5be2016 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:43.143796 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17398d27-79e8-40f0-85da-48ff9a7c3067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:43.144741 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17398d27-79e8-40f0-85da-48ff9a7c3067 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:43.145027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17398d27-79e8-40f0-85da-48ff9a7c3067 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:43.145307 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17398d27-79e8-40f0-85da-48ff9a7c3067 None admin] PATH_INFO: `/v3/projects/6a1c0f0b20b844d7b8311813151d93aa` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:43.146535 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17398d27-79e8-40f0-85da-48ff9a7c3067 None admin] RBAC: Authorizing `identity:get_project(project_id=6a1c0f0b20b844d7b8311813151d93aa)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:43.147978 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17398d27-79e8-40f0-85da-48ff9a7c3067 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:43.154208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 204/413] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:42 2022] GET /identity/v3/projects/6a1c0f0b20b844d7b8311813151d93aa => generated 344 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:43.154208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7dd40-ad29-4323-bb03-6652866946ee None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:43.154208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7dd40-ad29-4323-bb03-6652866946ee None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:43.154208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7dd40-ad29-4323-bb03-6652866946ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:43.169926 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6bc7dd40-ad29-4323-bb03-6652866946ee None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:43.171241 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6bc7dd40-ad29-4323-bb03-6652866946ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:43.230399 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199e898-5f2a-4d3b-be06-f126a5be2016 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:43.240336 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 205/414] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:43.241324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7dd40-ad29-4323-bb03-6652866946ee None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:43.274463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1fbf5-edb6-4a15-8844-6e7b2ba42e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:43.275512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1fbf5-edb6-4a15-8844-6e7b2ba42e78 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:43.275806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1fbf5-edb6-4a15-8844-6e7b2ba42e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:43.276392 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1fbf5-edb6-4a15-8844-6e7b2ba42e78 None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687/groups/95b3afd9130d4a7aad00b03a635a1f4f/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:43.278646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d1fbf5-edb6-4a15-8844-6e7b2ba42e78 None admin] RBAC: Authorizing `identity:create_grant(project_id=e87c7eebde3941f1aa5f40ce794e8687, group_id=95b3afd9130d4a7aad00b03a635a1f4f, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:43.281023 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d1fbf5-edb6-4a15-8844-6e7b2ba42e78 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:43.306771 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:43.343943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc7dd40-ad29-4323-bb03-6652866946ee None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:43.382251 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d18c7-f971-48af-988c-af3b80d88afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:43.383676 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d18c7-f971-48af-988c-af3b80d88afa None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:43.384101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d18c7-f971-48af-988c-af3b80d88afa None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:43.384520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d18c7-f971-48af-988c-af3b80d88afa None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687/groups/95b3afd9130d4a7aad00b03a635a1f4f/roles/a048041c27f44edf98a110d23138a327` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:43.387818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d18c7-f971-48af-988c-af3b80d88afa None admin] RBAC: Authorizing `identity:create_grant(project_id=e87c7eebde3941f1aa5f40ce794e8687, group_id=95b3afd9130d4a7aad00b03a635a1f4f, role_id=a048041c27f44edf98a110d23138a327)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:43.390095 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d18c7-f971-48af-988c-af3b80d88afa None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:43.399657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:43.412905 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 210/415] 10.222.0.16 () {58 vars in 1442 bytes} [Fri Jun 24 03:58:43 2022] PUT /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687/groups/95b3afd9130d4a7aad00b03a635a1f4f/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:58:43.412905 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1fbf5-edb6-4a15-8844-6e7b2ba42e78 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:43.474909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 206/416] 10.222.0.16 () {58 vars in 1442 bytes} [Fri Jun 24 03:58:43 2022] PUT /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687/groups/95b3afd9130d4a7aad00b03a635a1f4f/roles/a048041c27f44edf98a110d23138a327 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:58:43.476471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d18c7-f971-48af-988c-af3b80d88afa None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:43.578007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1fbf5-edb6-4a15-8844-6e7b2ba42e78 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:43.627071 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71332b-70b8-4385-97cf-0e4791461c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:43.628936 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71332b-70b8-4385-97cf-0e4791461c64 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:43.629423 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71332b-70b8-4385-97cf-0e4791461c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:43.629895 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71332b-70b8-4385-97cf-0e4791461c64 None admin] PATH_INFO: `/v3/projects/6a1c0f0b20b844d7b8311813151d93aa/groups/445579294e72451cbcc55238bc259ced/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:43.633845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71332b-70b8-4385-97cf-0e4791461c64 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a1c0f0b20b844d7b8311813151d93aa, group_id=445579294e72451cbcc55238bc259ced, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:43.635698 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71332b-70b8-4385-97cf-0e4791461c64 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:43.652264 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d18c7-f971-48af-988c-af3b80d88afa None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:43.680713 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:43.693116 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8250b-5742-48bf-87be-a3cd7336e823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:43.694153 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8250b-5742-48bf-87be-a3cd7336e823 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:43.694614 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8250b-5742-48bf-87be-a3cd7336e823 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:43.695088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8250b-5742-48bf-87be-a3cd7336e823 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:43.713476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 211/417] 10.222.0.16 () {58 vars in 1442 bytes} [Fri Jun 24 03:58:43 2022] PUT /identity/v3/projects/6a1c0f0b20b844d7b8311813151d93aa/groups/445579294e72451cbcc55238bc259ced/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:58:43.714267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8250b-5742-48bf-87be-a3cd7336e823 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:43.723579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-d9c8250b-5742-48bf-87be-a3cd7336e823 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:43.723579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:43.723579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:43.723579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:43.723579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:43.723579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:43.723579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:43.723579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:43.723579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:43.723579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:43.723579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:43.723579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:43.723579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:43.723579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:43.723579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:43.723579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:43.724734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:43.724734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:43.724734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:43.724734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:43.724734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:43.724734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:43.724734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:43.724734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:43.724734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:43.724734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:43.724734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:43.724734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:43.724734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:43.724734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:43.724734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:43.724734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:43.725833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:43.725833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:43.725833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:43.725833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:43.725833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:43.725833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:43.725833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:43.725833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:43.725833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:43.725833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:43.725833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:43.725833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:43.728594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 207/418] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:43.737339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71332b-70b8-4385-97cf-0e4791461c64 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:43.820425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:43.866773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71332b-70b8-4385-97cf-0e4791461c64 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:43.915405 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb31f58-6dd5-479c-aee8-a5bc7c1350b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:43.916072 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb31f58-6dd5-479c-aee8-a5bc7c1350b3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:43.916362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb31f58-6dd5-479c-aee8-a5bc7c1350b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:43.916641 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb31f58-6dd5-479c-aee8-a5bc7c1350b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:43.917400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb31f58-6dd5-479c-aee8-a5bc7c1350b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:43.917713 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb31f58-6dd5-479c-aee8-a5bc7c1350b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:43.919117 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb31f58-6dd5-479c-aee8-a5bc7c1350b3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:43.935220 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 212/419] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:43.941288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8250b-5742-48bf-87be-a3cd7336e823 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:43.967036 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb6350-2905-443e-85e6-ae26e601d2ce None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:43.967323 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb6350-2905-443e-85e6-ae26e601d2ce None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:43.967605 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb6350-2905-443e-85e6-ae26e601d2ce None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:43.972636 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 213/420] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:43.976557 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc106727-abd9-4f0b-9a17-70883894ca61 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:43.976850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc106727-abd9-4f0b-9a17-70883894ca61 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:43.977129 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc106727-abd9-4f0b-9a17-70883894ca61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:43.996824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-cc106727-abd9-4f0b-9a17-70883894ca61 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:44.001072 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-cc106727-abd9-4f0b-9a17-70883894ca61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:44.063805 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8250b-5742-48bf-87be-a3cd7336e823 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:44.082920 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 214/421] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:58:44.102401 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ef125-1abe-46fc-abdf-030e3f8a1291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:44.103067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ef125-1abe-46fc-abdf-030e3f8a1291 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:44.103353 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ef125-1abe-46fc-abdf-030e3f8a1291 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:44.103633 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ef125-1abe-46fc-abdf-030e3f8a1291 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:44.105751 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc106727-abd9-4f0b-9a17-70883894ca61 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:44.167495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:44.177365 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ef125-1abe-46fc-abdf-030e3f8a1291 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:44.178933 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ef125-1abe-46fc-abdf-030e3f8a1291 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:44.207328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:44.229088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc106727-abd9-4f0b-9a17-70883894ca61 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:44.267004 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3102e1c-4536-436c-a5de-0df6578f5cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:44.268121 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3102e1c-4536-436c-a5de-0df6578f5cfc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:44.268406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3102e1c-4536-436c-a5de-0df6578f5cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:44.268678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3102e1c-4536-436c-a5de-0df6578f5cfc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:44.282406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3102e1c-4536-436c-a5de-0df6578f5cfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:44.292634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-a3102e1c-4536-436c-a5de-0df6578f5cfc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:44.292634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:44.292634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:44.292634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:44.292634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:44.292634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:44.292634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:44.292634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:44.292634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:44.292634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:44.292634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:44.292634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:44.292634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:44.292634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:44.292634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:44.292634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:44.293591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:44.293591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:44.293591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:44.293591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:44.293591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:44.293591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:44.293591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:44.293591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:44.293591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:44.293591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:44.293591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:44.293591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:44.293591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:44.293591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:44.293591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:44.293591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:44.294417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:44.294417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:44.294417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:44.294417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:44.294417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:44.294417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:44.294417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:44.294417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:44.294417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:44.294417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:44.294417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:44.294417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:44.298966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 215/422] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:58:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:44.301150 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3102e1c-4536-436c-a5de-0df6578f5cfc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:44.324630 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 208/423] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:44.330156 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ef125-1abe-46fc-abdf-030e3f8a1291 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:44.423050 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3102e1c-4536-436c-a5de-0df6578f5cfc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:44.435265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:44.471869 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2dc81-5561-40ce-b0bb-6b85d9ccde19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:44.478389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2dc81-5561-40ce-b0bb-6b85d9ccde19 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:44.478389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2dc81-5561-40ce-b0bb-6b85d9ccde19 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:44.478389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2dc81-5561-40ce-b0bb-6b85d9ccde19 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:44.478389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2dc81-5561-40ce-b0bb-6b85d9ccde19 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:44.478389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2dc81-5561-40ce-b0bb-6b85d9ccde19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:44.478389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2dc81-5561-40ce-b0bb-6b85d9ccde19 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:44.490511 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 216/424] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:58:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:44.493267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2dc81-5561-40ce-b0bb-6b85d9ccde19 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:44.514056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:44.542052 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ef125-1abe-46fc-abdf-030e3f8a1291 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:44.618108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c19a29-176d-4348-806b-f033615fc19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:44.619238 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c19a29-176d-4348-806b-f033615fc19a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:44.619514 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c19a29-176d-4348-806b-f033615fc19a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:44.619785 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c19a29-176d-4348-806b-f033615fc19a None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:44.622083 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c19a29-176d-4348-806b-f033615fc19a None admin] RBAC: Authorizing `identity:get_project(project_id=40f51bfd8db04611b068f7edd7002004)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:44.623559 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c19a29-176d-4348-806b-f033615fc19a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:44.626946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 209/425] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:44 2022] GET /identity/v3/projects/40f51bfd8db04611b068f7edd7002004 => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:44.631487 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c19a29-176d-4348-806b-f033615fc19a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:44.667209 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2dc81-5561-40ce-b0bb-6b85d9ccde19 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:44.730746 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2778f40-f25c-46e9-b325-545753334274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:44.731317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2778f40-f25c-46e9-b325-545753334274 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:44.731601 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2778f40-f25c-46e9-b325-545753334274 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:44.731877 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2778f40-f25c-46e9-b325-545753334274 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:44.822764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:44.823081 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c19a29-176d-4348-806b-f033615fc19a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:44.868608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2778f40-f25c-46e9-b325-545753334274 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:44.870344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2778f40-f25c-46e9-b325-545753334274 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:44.898548 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447abfa2-bd04-4ec7-b662-e96ac531bf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:44.899406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447abfa2-bd04-4ec7-b662-e96ac531bf3f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:44.899682 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447abfa2-bd04-4ec7-b662-e96ac531bf3f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:44.899952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447abfa2-bd04-4ec7-b662-e96ac531bf3f None admin] PATH_INFO: `/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:44.901240 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447abfa2-bd04-4ec7-b662-e96ac531bf3f None admin] RBAC: Authorizing `identity:get_group(group_id=95b3afd9130d4a7aad00b03a635a1f4f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:44.902696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447abfa2-bd04-4ec7-b662-e96ac531bf3f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:44.906722 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 210/426] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jun 24 03:58:44 2022] GET /identity/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:44.911701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447abfa2-bd04-4ec7-b662-e96ac531bf3f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:44.951378 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:45.079057 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447abfa2-bd04-4ec7-b662-e96ac531bf3f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:45.085739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 217/427] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 03:58:45.095320 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2778f40-f25c-46e9-b325-545753334274 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:45.122993 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacf896-2e0f-44aa-ac64-f33e49b5f49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:45.123659 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacf896-2e0f-44aa-ac64-f33e49b5f49e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:45.123943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacf896-2e0f-44aa-ac64-f33e49b5f49e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:45.124222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacf896-2e0f-44aa-ac64-f33e49b5f49e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:45.125058 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacf896-2e0f-44aa-ac64-f33e49b5f49e None admin] RBAC: Adding query filter params (group.id=95b3afd9130d4a7aad00b03a635a1f4f, role.id=d7aadc20cd4c4838ae5d7d8e8826abae, scope.project.id=40f51bfd8db04611b068f7edd7002004) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:45.125375 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacf896-2e0f-44aa-ac64-f33e49b5f49e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:45.126845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacf896-2e0f-44aa-ac64-f33e49b5f49e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:45.145570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 211/428] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jun 24 03:58:44 2022] GET /identity/v3/role_assignments?group.id=95b3afd9130d4a7aad00b03a635a1f4f&scope.project.id=40f51bfd8db04611b068f7edd7002004&role.id=d7aadc20cd4c4838ae5d7d8e8826abae => generated 611 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:45.207361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:45.241088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2778f40-f25c-46e9-b325-545753334274 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:45.278707 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cfd8a-4e87-42b7-b91e-821b13428261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:45.279646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cfd8a-4e87-42b7-b91e-821b13428261 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:45.279937 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cfd8a-4e87-42b7-b91e-821b13428261 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:45.280221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227cfd8a-4e87-42b7-b91e-821b13428261 None admin] PATH_INFO: `/v3/projects/40f51bfd8db04611b068f7edd7002004` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:45.281533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cfd8a-4e87-42b7-b91e-821b13428261 None admin] RBAC: Authorizing `identity:get_project(project_id=40f51bfd8db04611b068f7edd7002004)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:45.283252 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227cfd8a-4e87-42b7-b91e-821b13428261 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:45.285914 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 218/429] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:45 2022] GET /identity/v3/projects/40f51bfd8db04611b068f7edd7002004 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:45.291400 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacf896-2e0f-44aa-ac64-f33e49b5f49e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:45.392704 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:45.413533 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacf896-2e0f-44aa-ac64-f33e49b5f49e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:45.461504 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8582116-24f5-4064-ab81-e5fb5bd41660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:45.462395 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8582116-24f5-4064-ab81-e5fb5bd41660 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:45.462697 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8582116-24f5-4064-ab81-e5fb5bd41660 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:45.462978 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8582116-24f5-4064-ab81-e5fb5bd41660 None admin] PATH_INFO: `/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:45.464182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8582116-24f5-4064-ab81-e5fb5bd41660 None admin] RBAC: Authorizing `identity:get_group(group_id=95b3afd9130d4a7aad00b03a635a1f4f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:45.465636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8582116-24f5-4064-ab81-e5fb5bd41660 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:45.467624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 212/430] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jun 24 03:58:45 2022] GET /identity/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:45.475607 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cfd8a-4e87-42b7-b91e-821b13428261 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:45.604035 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227cfd8a-4e87-42b7-b91e-821b13428261 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:45.644261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b4133-0caf-44a0-b52a-142410b40342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:45.645718 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b4133-0caf-44a0-b52a-142410b40342 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:45.646145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b4133-0caf-44a0-b52a-142410b40342 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:45.646594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b4133-0caf-44a0-b52a-142410b40342 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:45.647575 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b4133-0caf-44a0-b52a-142410b40342 None admin] RBAC: Adding query filter params (group.id=95b3afd9130d4a7aad00b03a635a1f4f, role.id=a048041c27f44edf98a110d23138a327, scope.project.id=40f51bfd8db04611b068f7edd7002004) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:45.648046 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b4133-0caf-44a0-b52a-142410b40342 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:45.649629 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b4133-0caf-44a0-b52a-142410b40342 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:45.662409 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 219/431] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jun 24 03:58:45 2022] GET /identity/v3/role_assignments?group.id=95b3afd9130d4a7aad00b03a635a1f4f&scope.project.id=40f51bfd8db04611b068f7edd7002004&role.id=a048041c27f44edf98a110d23138a327 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:45.833005 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1640a2-7150-4652-87ef-86de19328043 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:45.833583 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1640a2-7150-4652-87ef-86de19328043 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:45.834071 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1640a2-7150-4652-87ef-86de19328043 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:45.835980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 213/432] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:45.840975 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b498b3-1b98-46d6-8a43-d417e0c2d6c8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:45.841532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b498b3-1b98-46d6-8a43-d417e0c2d6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:45.842013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b498b3-1b98-46d6-8a43-d417e0c2d6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:45.850173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:45.864624 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c6b498b3-1b98-46d6-8a43-d417e0c2d6c8 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:45.866249 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c6b498b3-1b98-46d6-8a43-d417e0c2d6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:45.921668 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046dfd7-d50e-49b2-bcc6-9e861addd6d7 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:45.921942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046dfd7-d50e-49b2-bcc6-9e861addd6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:45.922156 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046dfd7-d50e-49b2-bcc6-9e861addd6d7 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:45.923858 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 214/433] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:45.932819 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8f1b7-3e66-4f10-a82c-960498295fc6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:45.932819 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8f1b7-3e66-4f10-a82c-960498295fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:45.933060 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc8f1b7-3e66-4f10-a82c-960498295fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:45.951459 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6fc8f1b7-3e66-4f10-a82c-960498295fc6 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:45.952739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6fc8f1b7-3e66-4f10-a82c-960498295fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:45.960260 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 220/434] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:45.983472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b498b3-1b98-46d6-8a43-d417e0c2d6c8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:46.034813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 215/435] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:58:46.058265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8f1b7-3e66-4f10-a82c-960498295fc6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:46.130293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b498b3-1b98-46d6-8a43-d417e0c2d6c8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:46.160569 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:46.184533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc90098-f126-41e4-a9b7-92ad58b4509b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:46.184533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc90098-f126-41e4-a9b7-92ad58b4509b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:46.184533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc90098-f126-41e4-a9b7-92ad58b4509b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:46.184533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc90098-f126-41e4-a9b7-92ad58b4509b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:46.185992 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:46.187172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc8f1b7-3e66-4f10-a82c-960498295fc6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:46.201624 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc90098-f126-41e4-a9b7-92ad58b4509b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:46.212292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-2bc90098-f126-41e4-a9b7-92ad58b4509b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:46.212292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:46.212292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:46.212292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:46.212292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:46.212292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:46.212292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:46.212292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:46.212292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:46.212292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:46.212292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:46.212292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:46.212292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:46.212292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:46.212292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:46.212292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:46.216703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:46.216703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:46.216703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:46.216703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:46.216703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:46.216703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:46.216703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:46.216703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:46.216703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:46.216703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:46.216703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:46.216703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:46.216703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:46.216703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:46.216703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:46.216703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:46.217603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:46.217603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:46.217603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:46.217603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:46.217603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:46.217603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:46.217603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:46.217603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:46.217603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:46.217603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:46.217603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 03:58:46.217603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:46.217603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 221/436] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:58:46.220008 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc90098-f126-41e4-a9b7-92ad58b4509b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:46.235490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769d89c-0d81-433a-a596-00e1eb26c1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:46.236310 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769d89c-0d81-433a-a596-00e1eb26c1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:46.236773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769d89c-0d81-433a-a596-00e1eb26c1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:46.237062 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7769d89c-0d81-433a-a596-00e1eb26c1a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:46.251277 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7769d89c-0d81-433a-a596-00e1eb26c1a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:46.260455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-7769d89c-0d81-433a-a596-00e1eb26c1a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:46.260455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:46.260455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:46.260455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:46.260455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:46.260455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:46.260455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:46.260455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:46.260455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:46.260455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:46.260455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:46.260455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:46.260455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:46.260455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:46.260455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:46.260455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:46.261370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:46.261370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:46.261370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:46.261370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:46.261370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:46.261370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:46.261370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:46.261370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:46.261370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:46.261370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:46.261370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:46.261370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:46.261370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:46.261370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:46.261370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:46.261370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:46.262208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:46.262208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:46.262208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:46.262208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:46.262208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:46.262208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:46.262208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:46.262208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:46.262208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:46.262208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:46.262208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:46.262208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:46.263388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 216/437] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:58:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:58:46.271724 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769d89c-0d81-433a-a596-00e1eb26c1a3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:46.350554 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc90098-f126-41e4-a9b7-92ad58b4509b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:46.386985 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7769d89c-0d81-433a-a596-00e1eb26c1a3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:46.387494 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd746919-d552-4ed9-bcb7-34eb8afb2b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:46.388171 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd746919-d552-4ed9-bcb7-34eb8afb2b30 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:46.388625 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd746919-d552-4ed9-bcb7-34eb8afb2b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:46.388910 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd746919-d552-4ed9-bcb7-34eb8afb2b30 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:46.390494 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd746919-d552-4ed9-bcb7-34eb8afb2b30 None admin] RBAC: Adding query filter params (name=member) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:46.390821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd746919-d552-4ed9-bcb7-34eb8afb2b30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:46.392216 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd746919-d552-4ed9-bcb7-34eb8afb2b30 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:46.402320 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 222/438] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:58:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:46.407499 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd746919-d552-4ed9-bcb7-34eb8afb2b30 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:46.426897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5009847-6767-4a2f-96a4-acad3ffb3926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:46.427736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5009847-6767-4a2f-96a4-acad3ffb3926 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:46.428018 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5009847-6767-4a2f-96a4-acad3ffb3926 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:46.428603 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5009847-6767-4a2f-96a4-acad3ffb3926 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:46.429358 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5009847-6767-4a2f-96a4-acad3ffb3926 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:46.429674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5009847-6767-4a2f-96a4-acad3ffb3926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:46.431530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5009847-6767-4a2f-96a4-acad3ffb3926 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:46.440134 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 217/439] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:58:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:46.446004 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5009847-6767-4a2f-96a4-acad3ffb3926 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:46.448724 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:46.468846 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:46.512945 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd746919-d552-4ed9-bcb7-34eb8afb2b30 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:46.545477 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5009847-6767-4a2f-96a4-acad3ffb3926 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:46.545632 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f27b69-c475-4628-9b72-49faaf4f006a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:46.546272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f27b69-c475-4628-9b72-49faaf4f006a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:46.546591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f27b69-c475-4628-9b72-49faaf4f006a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:46.546877 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f27b69-c475-4628-9b72-49faaf4f006a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:46.578959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d606589-5e8f-4076-9ddd-8649f1a66c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:46.579601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d606589-5e8f-4076-9ddd-8649f1a66c66 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:46.579887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d606589-5e8f-4076-9ddd-8649f1a66c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:46.580170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d606589-5e8f-4076-9ddd-8649f1a66c66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:46.613320 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f27b69-c475-4628-9b72-49faaf4f006a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:46.615449 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f27b69-c475-4628-9b72-49faaf4f006a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:46.651848 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d606589-5e8f-4076-9ddd-8649f1a66c66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:46.653296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d606589-5e8f-4076-9ddd-8649f1a66c66 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:46.655155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:46.670872 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:46.719314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 223/440] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:46.719314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ae936-a65d-4531-a097-3e7e98acfb8e None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:46.719314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ae936-a65d-4531-a097-3e7e98acfb8e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:46.719314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ae936-a65d-4531-a097-3e7e98acfb8e None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:46.719314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 224/441] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:46.720261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ae936-a65d-4531-a097-3e7e98acfb8e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:46.757532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 218/442] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 03:58:46.758284 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990a41a-1c14-4fcc-a9e1-e18122458362 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:46.758603 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990a41a-1c14-4fcc-a9e1-e18122458362 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:46.758884 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990a41a-1c14-4fcc-a9e1-e18122458362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:46.775007 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3990a41a-1c14-4fcc-a9e1-e18122458362 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:46.776194 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3990a41a-1c14-4fcc-a9e1-e18122458362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:46.822589 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ae936-a65d-4531-a097-3e7e98acfb8e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:46.843907 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 219/443] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:46.845025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990a41a-1c14-4fcc-a9e1-e18122458362 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:46.862548 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abd15a-0efa-4a62-be38-aa4d14e557f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:46.863447 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abd15a-0efa-4a62-be38-aa4d14e557f1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:46.863732 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abd15a-0efa-4a62-be38-aa4d14e557f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:46.864009 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6abd15a-0efa-4a62-be38-aa4d14e557f1 None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:46.865259 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abd15a-0efa-4a62-be38-aa4d14e557f1 None admin] RBAC: Authorizing `identity:get_project(project_id=e87c7eebde3941f1aa5f40ce794e8687)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:46.866747 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6abd15a-0efa-4a62-be38-aa4d14e557f1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:46.868437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 225/444] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:46 2022] GET /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:46.869456 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abd15a-0efa-4a62-be38-aa4d14e557f1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:46.881731 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:46.895617 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:46.950175 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990a41a-1c14-4fcc-a9e1-e18122458362 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:46.969936 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6abd15a-0efa-4a62-be38-aa4d14e557f1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:46.984196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d81a3f-0a80-4aad-a5d2-37ea8db7a7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:46.985047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d81a3f-0a80-4aad-a5d2-37ea8db7a7d6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:46.985324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d81a3f-0a80-4aad-a5d2-37ea8db7a7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:46.985675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d81a3f-0a80-4aad-a5d2-37ea8db7a7d6 None admin] PATH_INFO: `/v3/projects/e87c7eebde3941f1aa5f40ce794e8687` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:46.986992 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d81a3f-0a80-4aad-a5d2-37ea8db7a7d6 None admin] RBAC: Authorizing `identity:get_project(project_id=e87c7eebde3941f1aa5f40ce794e8687)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:46.988917 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d81a3f-0a80-4aad-a5d2-37ea8db7a7d6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:46.990683 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 220/445] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:46 2022] GET /identity/v3/projects/e87c7eebde3941f1aa5f40ce794e8687 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:46.991556 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d81a3f-0a80-4aad-a5d2-37ea8db7a7d6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:47.009155 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a85482-51f2-4c23-ab40-4e9063b0870b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:47.009947 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a85482-51f2-4c23-ab40-4e9063b0870b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:47.010239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a85482-51f2-4c23-ab40-4e9063b0870b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:47.010561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a85482-51f2-4c23-ab40-4e9063b0870b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:47.021107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a85482-51f2-4c23-ab40-4e9063b0870b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:47.028787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-83a85482-51f2-4c23-ab40-4e9063b0870b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:58:47.028787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:47.028787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:47.028787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:47.028787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:47.028787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:47.028787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:47.028787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:47.028787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:47.028787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:47.028787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:47.028787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:47.028787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:47.028787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:47.028787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:47.028787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:47.029661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:47.029661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:47.029661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:47.029661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:47.029661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:47.029661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:47.029661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:47.029661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:47.029661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:47.029661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:47.029661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:47.029661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:47.029661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:47.029661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:47.029661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:47.029661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:47.030522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:47.030522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:47.030522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:47.030522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:47.030522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:47.030522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:47.030522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:47.030522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:47.030522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:47.030522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:47.030522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:58:47.030522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:47.030522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 226/446] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:58:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:47.031191 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a85482-51f2-4c23-ab40-4e9063b0870b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:47.094288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:47.097275 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d81a3f-0a80-4aad-a5d2-37ea8db7a7d6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:47.115334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:47.131281 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a85482-51f2-4c23-ab40-4e9063b0870b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:47.131863 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c16484-d3cc-406b-a09a-282ef1743480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:47.132392 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c16484-d3cc-406b-a09a-282ef1743480 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:47.132677 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c16484-d3cc-406b-a09a-282ef1743480 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:47.132949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c16484-d3cc-406b-a09a-282ef1743480 None admin] PATH_INFO: `/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:47.134139 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c16484-d3cc-406b-a09a-282ef1743480 None admin] RBAC: Authorizing `identity:get_group(group_id=95b3afd9130d4a7aad00b03a635a1f4f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:47.135621 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c16484-d3cc-406b-a09a-282ef1743480 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:47.137240 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 221/447] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jun 24 03:58:46 2022] GET /identity/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:47.137996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c16484-d3cc-406b-a09a-282ef1743480 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:47.165630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bafc0-23f1-44b7-9186-891a31c6ee0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:47.166582 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bafc0-23f1-44b7-9186-891a31c6ee0b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:47.166867 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bafc0-23f1-44b7-9186-891a31c6ee0b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:47.167144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bafc0-23f1-44b7-9186-891a31c6ee0b None admin] PATH_INFO: `/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:47.168307 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bafc0-23f1-44b7-9186-891a31c6ee0b None admin] RBAC: Authorizing `identity:get_group(group_id=95b3afd9130d4a7aad00b03a635a1f4f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:47.169746 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bafc0-23f1-44b7-9186-891a31c6ee0b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:47.171541 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 227/448] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jun 24 03:58:47 2022] GET /identity/v3/groups/95b3afd9130d4a7aad00b03a635a1f4f => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:47.172412 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bafc0-23f1-44b7-9186-891a31c6ee0b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:47.237935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c16484-d3cc-406b-a09a-282ef1743480 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:47.272783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e34ad5-792f-42db-a900-7dc76094dec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:47.273437 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e34ad5-792f-42db-a900-7dc76094dec5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:47.273729 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e34ad5-792f-42db-a900-7dc76094dec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:47.274009 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e34ad5-792f-42db-a900-7dc76094dec5 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:47.274796 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e34ad5-792f-42db-a900-7dc76094dec5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:47.275108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e34ad5-792f-42db-a900-7dc76094dec5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:47.276477 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e34ad5-792f-42db-a900-7dc76094dec5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:47.276635 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bafc0-23f1-44b7-9186-891a31c6ee0b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:47.284903 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 222/449] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:47.285682 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e34ad5-792f-42db-a900-7dc76094dec5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:47.314593 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5757c7-36f1-4551-97ed-196251bbec84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:47.315249 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5757c7-36f1-4551-97ed-196251bbec84 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:47.315353 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:47.316140 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5757c7-36f1-4551-97ed-196251bbec84 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:47.316439 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5757c7-36f1-4551-97ed-196251bbec84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:47.317272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5757c7-36f1-4551-97ed-196251bbec84 None admin] RBAC: Adding query filter params (group.id=95b3afd9130d4a7aad00b03a635a1f4f, role.id=d7aadc20cd4c4838ae5d7d8e8826abae, scope.project.id=e87c7eebde3941f1aa5f40ce794e8687) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:47.317586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5757c7-36f1-4551-97ed-196251bbec84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:47.319063 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5757c7-36f1-4551-97ed-196251bbec84 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:47.324973 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 228/450] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jun 24 03:58:47 2022] GET /identity/v3/role_assignments?group.id=95b3afd9130d4a7aad00b03a635a1f4f&scope.project.id=e87c7eebde3941f1aa5f40ce794e8687&role.id=d7aadc20cd4c4838ae5d7d8e8826abae => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:47.325755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5757c7-36f1-4551-97ed-196251bbec84 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:47.357479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:47.395511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e34ad5-792f-42db-a900-7dc76094dec5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:47.433798 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f5156-bfec-4d63-9f4d-b73f08ff451f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:47.434660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f5156-bfec-4d63-9f4d-b73f08ff451f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:47.434949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f5156-bfec-4d63-9f4d-b73f08ff451f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:47.435229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f5156-bfec-4d63-9f4d-b73f08ff451f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:47.436072 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f5156-bfec-4d63-9f4d-b73f08ff451f None admin] RBAC: Adding query filter params (group.id=95b3afd9130d4a7aad00b03a635a1f4f, role.id=a048041c27f44edf98a110d23138a327, scope.project.id=e87c7eebde3941f1aa5f40ce794e8687) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:47.436390 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f5156-bfec-4d63-9f4d-b73f08ff451f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:47.437868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8f5156-bfec-4d63-9f4d-b73f08ff451f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:47.454412 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 223/451] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jun 24 03:58:47 2022] GET /identity/v3/role_assignments?group.id=95b3afd9130d4a7aad00b03a635a1f4f&scope.project.id=e87c7eebde3941f1aa5f40ce794e8687&role.id=a048041c27f44edf98a110d23138a327 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:47.466802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5757c7-36f1-4551-97ed-196251bbec84 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:47.511824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052da45-ae94-476c-8075-e52ad2db3578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:47.512423 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052da45-ae94-476c-8075-e52ad2db3578 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:47.512712 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052da45-ae94-476c-8075-e52ad2db3578 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:47.512996 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052da45-ae94-476c-8075-e52ad2db3578 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:47.596666 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052da45-ae94-476c-8075-e52ad2db3578 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:47.598552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052da45-ae94-476c-8075-e52ad2db3578 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:47.618047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:47.705208 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 229/452] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:47.711800 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f5156-bfec-4d63-9f4d-b73f08ff451f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:47.811016 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:47.818556 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f5156-bfec-4d63-9f4d-b73f08ff451f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:47.852905 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3abfe-a968-497f-8cf1-94e26cedeb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:47.853764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3abfe-a968-497f-8cf1-94e26cedeb3e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:47.854045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3abfe-a968-497f-8cf1-94e26cedeb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:47.854318 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3abfe-a968-497f-8cf1-94e26cedeb3e None admin] PATH_INFO: `/v3/projects/6a1c0f0b20b844d7b8311813151d93aa` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:47.855524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3abfe-a968-497f-8cf1-94e26cedeb3e None admin] RBAC: Authorizing `identity:get_project(project_id=6a1c0f0b20b844d7b8311813151d93aa)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:47.856953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3abfe-a968-497f-8cf1-94e26cedeb3e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:47.858653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 224/453] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 03:58:47 2022] GET /identity/v3/projects/6a1c0f0b20b844d7b8311813151d93aa => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:47.865327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052da45-ae94-476c-8075-e52ad2db3578 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:47.962443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052da45-ae94-476c-8075-e52ad2db3578 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:47.977930 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:47.995722 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17e5037-3cc0-407a-8d63-8393209c9337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:47.996579 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e5037-3cc0-407a-8d63-8393209c9337 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:47.996865 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e5037-3cc0-407a-8d63-8393209c9337 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:47.997142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17e5037-3cc0-407a-8d63-8393209c9337 None admin] PATH_INFO: `/v3/groups/445579294e72451cbcc55238bc259ced` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:47.998306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e5037-3cc0-407a-8d63-8393209c9337 None admin] RBAC: Authorizing `identity:get_group(group_id=445579294e72451cbcc55238bc259ced)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:47.999783 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17e5037-3cc0-407a-8d63-8393209c9337 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:48.001375 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 230/454] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Jun 24 03:58:47 2022] GET /identity/v3/groups/445579294e72451cbcc55238bc259ced => generated 229 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:48.008252 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3abfe-a968-497f-8cf1-94e26cedeb3e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:48.106439 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3abfe-a968-497f-8cf1-94e26cedeb3e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:48.139393 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216aa879-2dd7-4c63-b68d-3790720b49db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:48.140039 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aa879-2dd7-4c63-b68d-3790720b49db None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:48.140326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aa879-2dd7-4c63-b68d-3790720b49db None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:48.140609 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216aa879-2dd7-4c63-b68d-3790720b49db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:48.141431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216aa879-2dd7-4c63-b68d-3790720b49db None admin] RBAC: Adding query filter params (group.id=445579294e72451cbcc55238bc259ced, role.id=f0f7e28cbe314921afdfb657b163436f, scope.project.id=6a1c0f0b20b844d7b8311813151d93aa) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:48.141751 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216aa879-2dd7-4c63-b68d-3790720b49db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:48.143208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216aa879-2dd7-4c63-b68d-3790720b49db None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:48.149047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 225/455] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Jun 24 03:58:48 2022] GET /identity/v3/role_assignments?group.id=445579294e72451cbcc55238bc259ced&scope.project.id=6a1c0f0b20b844d7b8311813151d93aa&role.id=f0f7e28cbe314921afdfb657b163436f => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:50.589943 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89edda9-782c-42dc-a50f-03de4d305cf0 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:50.590861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89edda9-782c-42dc-a50f-03de4d305cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:50.591288 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89edda9-782c-42dc-a50f-03de4d305cf0 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:50.593046 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 231/456] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:50.608316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afc4dd-6422-4e6d-a6e9-f1ed3c7324cb None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:50.609161 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afc4dd-6422-4e6d-a6e9-f1ed3c7324cb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:50.609617 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afc4dd-6422-4e6d-a6e9-f1ed3c7324cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:50.635439 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e4afc4dd-6422-4e6d-a6e9-f1ed3c7324cb None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:50.636628 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e4afc4dd-6422-4e6d-a6e9-f1ed3c7324cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:50.658726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:50.724302 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 226/457] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:50.765538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89edda9-782c-42dc-a50f-03de4d305cf0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:50.899374 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89edda9-782c-42dc-a50f-03de4d305cf0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:50.940009 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc44b91-1fdb-4a1e-be52-3c3d60362286 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:50.940301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc44b91-1fdb-4a1e-be52-3c3d60362286 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:50.941182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc44b91-1fdb-4a1e-be52-3c3d60362286 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:50.942802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 227/458] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:50.946682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebef55-abc8-44fc-9103-3dcac290ffc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:50.947485 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebef55-abc8-44fc-9103-3dcac290ffc4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:50.947855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebef55-abc8-44fc-9103-3dcac290ffc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:50.948139 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebef55-abc8-44fc-9103-3dcac290ffc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:50.961182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e1f18-702e-4edf-abc4-60e4adf6f8ed None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:50.961738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e1f18-702e-4edf-abc4-60e4adf6f8ed None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:50.962081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebef55-abc8-44fc-9103-3dcac290ffc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:50.962452 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e1f18-702e-4edf-abc4-60e4adf6f8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:50.963703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebef55-abc8-44fc-9103-3dcac290ffc4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:50.990885 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-315e1f18-702e-4edf-abc4-60e4adf6f8ed None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:50.992435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-315e1f18-702e-4edf-abc4-60e4adf6f8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:50.994241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-f8ebef55-abc8-44fc-9103-3dcac290ffc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:50.994241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:50.994241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:50.994241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:50.994241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:50.994241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:50.994241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:50.994241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:50.994241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:50.994241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:50.994241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:50.994241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:50.994241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:58:50.994241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:58:50.994241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:58:50.995120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:58:50.995120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:50.995120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:50.995120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:50.995120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:50.995120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:50.995120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:50.995120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:50.995120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:50.995120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:50.995120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:50.995120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:50.995120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:50.995120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:50.995120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:50.995120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:50.995120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:50.995994 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:58:50.995994 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:58:50.995994 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:50.995994 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:50.995994 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 232/459] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:50.997138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07faeb0e-9f29-499d-abf1-082e996f01c7 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:50.997425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07faeb0e-9f29-499d-abf1-082e996f01c7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:50.997699 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07faeb0e-9f29-499d-abf1-082e996f01c7 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:50.998996 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 233/460] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:51.004865 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07faeb0e-9f29-499d-abf1-082e996f01c7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:51.043766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:51.079122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 228/461] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:51.080789 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cf1c7-32a5-4b7c-8afc-96623f41cd44 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:51.081312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cf1c7-32a5-4b7c-8afc-96623f41cd44 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:51.081792 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cf1c7-32a5-4b7c-8afc-96623f41cd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:51.098834 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-276cf1c7-32a5-4b7c-8afc-96623f41cd44 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:51.100352 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-276cf1c7-32a5-4b7c-8afc-96623f41cd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:51.116270 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07faeb0e-9f29-499d-abf1-082e996f01c7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:51.153206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d939cc-9c45-45ab-b4e9-ee121cf6d242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:51.154120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d939cc-9c45-45ab-b4e9-ee121cf6d242 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:51.154572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d939cc-9c45-45ab-b4e9-ee121cf6d242 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:51.155001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d939cc-9c45-45ab-b4e9-ee121cf6d242 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:51.155880 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d939cc-9c45-45ab-b4e9-ee121cf6d242 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:51.156320 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d939cc-9c45-45ab-b4e9-ee121cf6d242 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:51.157856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d939cc-9c45-45ab-b4e9-ee121cf6d242 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:51.171012 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 234/462] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:51.172045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d939cc-9c45-45ab-b4e9-ee121cf6d242 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:51.179650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 229/463] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:51.186388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cf1c7-32a5-4b7c-8afc-96623f41cd44 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:51.199001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:51.286811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:51.292169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cf1c7-32a5-4b7c-8afc-96623f41cd44 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:51.292479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d939cc-9c45-45ab-b4e9-ee121cf6d242 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:51.329509 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8c2dc-dfed-4c5b-abe4-00909868a41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:51.330584 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8c2dc-dfed-4c5b-abe4-00909868a41a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:51.330733 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d596ca-8e4d-4757-8acb-66d46c944ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:51.330919 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8c2dc-dfed-4c5b-abe4-00909868a41a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:51.331717 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8c2dc-dfed-4c5b-abe4-00909868a41a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:51.331936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d596ca-8e4d-4757-8acb-66d46c944ff5 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:51.332213 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d596ca-8e4d-4757-8acb-66d46c944ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:51.332495 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d596ca-8e4d-4757-8acb-66d46c944ff5 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:51.333207 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d596ca-8e4d-4757-8acb-66d46c944ff5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:51.334886 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d596ca-8e4d-4757-8acb-66d46c944ff5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:51.342689 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8c2dc-dfed-4c5b-abe4-00909868a41a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:51.344502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8c2dc-dfed-4c5b-abe4-00909868a41a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:51.362126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-0fa8c2dc-dfed-4c5b-abe4-00909868a41a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:51.362126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:51.362126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:51.362126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:51.362126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:51.362126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:51.362126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:51.362126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:51.362126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:51.362126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:51.362126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:51.362126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:51.362126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:58:51.362126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:58:51.362126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:58:51.363118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:58:51.363118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:51.363118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:51.363118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:51.363118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:51.363118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:51.363118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:51.363118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:51.363118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:51.363118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:51.363118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:51.363118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:51.363118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:51.363118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:51.363118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:51.363118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:51.363118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:51.364007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:58:51.364007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:58:51.364007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:51.364007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:51.364007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 235/464] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:51.364268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8c2dc-dfed-4c5b-abe4-00909868a41a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:51.438494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 230/465] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jun 24 03:58:51 2022] POST /identity/v3/users => generated 246 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:58:51.440031 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5aea6a-450f-4685-9349-8f330a2d0c7c None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:51.441072 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5aea6a-450f-4685-9349-8f330a2d0c7c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:51.441730 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5aea6a-450f-4685-9349-8f330a2d0c7c None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:51.442867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 231/466] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:51.443445 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5aea6a-450f-4685-9349-8f330a2d0c7c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:51.488583 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8c2dc-dfed-4c5b-abe4-00909868a41a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:51.543787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8301f-bc0c-45b4-a04c-4a149794590a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:51.545003 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8301f-bc0c-45b4-a04c-4a149794590a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:51.545368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8301f-bc0c-45b4-a04c-4a149794590a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:51.545719 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8301f-bc0c-45b4-a04c-4a149794590a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:51.547536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:51.561776 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8301f-bc0c-45b4-a04c-4a149794590a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:51.563640 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8301f-bc0c-45b4-a04c-4a149794590a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:51.577270 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:51.587472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-96d8301f-bc0c-45b4-a04c-4a149794590a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:51.587472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:51.587472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:51.587472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:51.587472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:51.587472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:51.587472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:51.587472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:51.587472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:51.587472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:51.587472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:51.587472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:51.587472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:58:51.587472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:58:51.587472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:58:51.588417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:58:51.588417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:51.588417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:51.588417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:51.588417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:51.588417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:51.588417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:51.588417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:51.588417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:51.588417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:51.588417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:51.588417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:51.588417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:51.588417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:51.588417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:51.588417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:51.588417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:51.589321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:58:51.589321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:58:51.589321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:51.589321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:51.591261 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5aea6a-450f-4685-9349-8f330a2d0c7c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:51.592509 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 236/467] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:51.593926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41b5f8-ccc8-49f6-9281-129cc07c846c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:51.594219 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41b5f8-ccc8-49f6-9281-129cc07c846c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:51.594533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d41b5f8-ccc8-49f6-9281-129cc07c846c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:51.612896 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0d41b5f8-ccc8-49f6-9281-129cc07c846c None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:51.615017 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0d41b5f8-ccc8-49f6-9281-129cc07c846c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:51.639165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d46c5-c7f4-4b7e-8500-fdcd59c0cc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:51.639842 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d46c5-c7f4-4b7e-8500-fdcd59c0cc56 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:51.640136 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d46c5-c7f4-4b7e-8500-fdcd59c0cc56 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:51.640419 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d46c5-c7f4-4b7e-8500-fdcd59c0cc56 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:51.641159 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d46c5-c7f4-4b7e-8500-fdcd59c0cc56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:51.641472 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d46c5-c7f4-4b7e-8500-fdcd59c0cc56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:51.642900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d46c5-c7f4-4b7e-8500-fdcd59c0cc56 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:51.657291 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 232/468] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:51.658130 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d46c5-c7f4-4b7e-8500-fdcd59c0cc56 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:51.714572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 237/469] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:51.714572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41b5f8-ccc8-49f6-9281-129cc07c846c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:51.800982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d46c5-c7f4-4b7e-8500-fdcd59c0cc56 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:51.821216 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d41b5f8-ccc8-49f6-9281-129cc07c846c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:51.835703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:51.841305 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ee586-a1d0-422e-9b51-aa149b2c613f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:51.841984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ee586-a1d0-422e-9b51-aa149b2c613f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:51.842269 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ee586-a1d0-422e-9b51-aa149b2c613f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:51.842581 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ee586-a1d0-422e-9b51-aa149b2c613f None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:51.843377 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ee586-a1d0-422e-9b51-aa149b2c613f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:51.843793 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ee586-a1d0-422e-9b51-aa149b2c613f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:51.845206 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35ee586-a1d0-422e-9b51-aa149b2c613f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:51.852953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:51.858717 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 233/470] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:58:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:51.858844 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff615b4-4809-487e-b1c6-137f0fa0737d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:51.859482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff615b4-4809-487e-b1c6-137f0fa0737d None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:51.859657 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ee586-a1d0-422e-9b51-aa149b2c613f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:51.859811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff615b4-4809-487e-b1c6-137f0fa0737d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:51.860174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff615b4-4809-487e-b1c6-137f0fa0737d None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:51.860895 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff615b4-4809-487e-b1c6-137f0fa0737d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:51.862993 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff615b4-4809-487e-b1c6-137f0fa0737d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:51.967714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35ee586-a1d0-422e-9b51-aa149b2c613f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:51.970748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 238/471] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jun 24 03:58:51 2022] POST /identity/v3/users => generated 248 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:58:51.972106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff615b4-4809-487e-b1c6-137f0fa0737d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:52.035332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b24f7-d385-4a82-bbad-ee72afaffa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:52.036134 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b24f7-d385-4a82-bbad-ee72afaffa9d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:52.036422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b24f7-d385-4a82-bbad-ee72afaffa9d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:52.036703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b24f7-d385-4a82-bbad-ee72afaffa9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:52.071699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b24f7-d385-4a82-bbad-ee72afaffa9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:52.076856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b24f7-d385-4a82-bbad-ee72afaffa9d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:52.119468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-910b24f7-d385-4a82-bbad-ee72afaffa9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:52.119468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:52.119468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:52.119468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:52.119468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:52.119468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:52.119468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:52.119468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:52.119468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:52.119468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:52.119468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:52.119468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:52.119468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:58:52.119468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:58:52.119468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:58:52.120447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:58:52.120447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:52.120447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:52.120447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:52.120447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:52.120447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:52.120447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:52.120447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:52.120447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:52.120447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:52.120447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:52.120447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:52.120447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:52.120447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:52.120447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:52.120447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:52.120447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:52.121322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:58:52.121322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:58:52.121322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:52.121322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:52.121322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 234/472] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:52.131456 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b24f7-d385-4a82-bbad-ee72afaffa9d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:52.138556 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff615b4-4809-487e-b1c6-137f0fa0737d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:52.181165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a676e3b-1a3b-4846-a0b2-e15e1bbab9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:52.182235 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a676e3b-1a3b-4846-a0b2-e15e1bbab9ed None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:52.182754 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a676e3b-1a3b-4846-a0b2-e15e1bbab9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:52.183717 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a676e3b-1a3b-4846-a0b2-e15e1bbab9ed None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:52.184651 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a676e3b-1a3b-4846-a0b2-e15e1bbab9ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:52.186349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a676e3b-1a3b-4846-a0b2-e15e1bbab9ed None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:52.234153 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:52.291019 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b24f7-d385-4a82-bbad-ee72afaffa9d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:52.298545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 239/473] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jun 24 03:58:51 2022] POST /identity/v3/users => generated 248 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:58:52.300213 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d729b-3208-466c-bf2a-7d72fa250778 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:52.300508 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d729b-3208-466c-bf2a-7d72fa250778 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:52.300788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d729b-3208-466c-bf2a-7d72fa250778 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:52.302502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 240/474] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:52.308521 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2ea16-174f-4c5e-bb3d-e35032d96a28 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:52.308815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2ea16-174f-4c5e-bb3d-e35032d96a28 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:52.309457 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2ea16-174f-4c5e-bb3d-e35032d96a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:52.334339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-28c2ea16-174f-4c5e-bb3d-e35032d96a28 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:52.337181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-28c2ea16-174f-4c5e-bb3d-e35032d96a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:52.346158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f48c11-bdba-400f-bae8-a677d8345830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:52.347666 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f48c11-bdba-400f-bae8-a677d8345830 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:52.347995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f48c11-bdba-400f-bae8-a677d8345830 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:52.348335 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f48c11-bdba-400f-bae8-a677d8345830 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:52.349086 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f48c11-bdba-400f-bae8-a677d8345830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:52.349950 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f48c11-bdba-400f-bae8-a677d8345830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:52.350988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f48c11-bdba-400f-bae8-a677d8345830 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:52.366398 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 235/475] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:52.373546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f48c11-bdba-400f-bae8-a677d8345830 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:52.398089 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:52.441210 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 241/476] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:52.500134 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2ea16-174f-4c5e-bb3d-e35032d96a28 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:52.512706 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f48c11-bdba-400f-bae8-a677d8345830 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:52.516931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:52.567046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d27c51-b5ce-4479-bc07-9816c9165a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:52.567727 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d27c51-b5ce-4479-bc07-9816c9165a3a None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:52.568011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d27c51-b5ce-4479-bc07-9816c9165a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:52.568290 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d27c51-b5ce-4479-bc07-9816c9165a3a None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:52.569009 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d27c51-b5ce-4479-bc07-9816c9165a3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:52.570524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d27c51-b5ce-4479-bc07-9816c9165a3a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:52.626655 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2ea16-174f-4c5e-bb3d-e35032d96a28 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:52.653918 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 236/477] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jun 24 03:58:52 2022] POST /identity/v3/users => generated 249 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 03:58:52.670494 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4407c-a4c5-4cbb-a7f8-5e245941f1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:52.671240 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4407c-a4c5-4cbb-a7f8-5e245941f1ed None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:52.671547 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4407c-a4c5-4cbb-a7f8-5e245941f1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:52.671810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4407c-a4c5-4cbb-a7f8-5e245941f1ed None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:52.672582 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4407c-a4c5-4cbb-a7f8-5e245941f1ed None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:52.673965 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4407c-a4c5-4cbb-a7f8-5e245941f1ed None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:52.716228 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.sql.core [None req-59d4407c-a4c5-4cbb-a7f8-5e245941f1ed None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 03:58:52.716228 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 03:58:52.716228 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [parameters: {'id': '73006f5405c94c4ab4adb57c2bd472b5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 03:58:52.716228 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80599) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 03:58:52.739194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-59d4407c-a4c5-4cbb-a7f8-5e245941f1ed None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 03:58:52.739194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:52.739194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 03:58:52.739194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 03:58:52.739194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 03:58:52.739194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 03:58:52.739194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 03:58:52.739194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 03:58:52.739194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 03:58:52.739194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application conn.query(q) Jun 24 03:58:52.739194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 03:58:52.739194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 03:58:52.739194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 03:58:52.739194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application result.read() Jun 24 03:58:52.739194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 03:58:52.740144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 03:58:52.740144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 03:58:52.740144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 03:58:52.740144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 03:58:52.740144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 03:58:52.740144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 03:58:52.740144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 03:58:52.740144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 03:58:52.740144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:52.740144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 03:58:52.740144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:52.740144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:52.740144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 03:58:52.740144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 03:58:52.740144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 24 03:58:52.740144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return ref.to_dict() Jun 24 03:58:52.740144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 03:58:52.740144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application next(self.gen) Jun 24 03:58:52.741088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 03:58:52.741088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application yield resource Jun 24 03:58:52.741088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 03:58:52.741088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application next(self.gen) Jun 24 03:58:52.741088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 03:58:52.741088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 03:58:52.741088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 03:58:52.741088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 03:58:52.741088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 03:58:52.741088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise self.value Jun 24 03:58:52.741088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 03:58:52.741088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 03:58:52.741088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 03:58:52.741088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application session.commit() Jun 24 03:58:52.741088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 03:58:52.741088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 03:58:52.741088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 03:58:52.741979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 03:58:52.741979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 03:58:52.741979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self.session.flush() Jun 24 03:58:52.741979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 03:58:52.741979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 03:58:52.741979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 03:58:52.741979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 03:58:52.741979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 03:58:52.741979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application compat.raise_( Jun 24 03:58:52.741979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 03:58:52.741979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception Jun 24 03:58:52.741979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 03:58:52.741979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 03:58:52.741979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 03:58:52.741979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 03:58:52.741979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 03:58:52.741979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 03:58:52.742963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 03:58:52.742963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 03:58:52.742963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 03:58:52.742963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 03:58:52.742963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 03:58:52.742963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 03:58:52.742963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 03:58:52.742963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 03:58:52.742963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 03:58:52.742963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 03:58:52.742963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 03:58:52.742963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 03:58:52.742963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 03:58:52.742963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 03:58:52.742963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 03:58:52.742963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception Jun 24 03:58:52.744095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 03:58:52.744095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 03:58:52.744095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 03:58:52.744095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 03:58:52.744095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 03:58:52.744095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 03:58:52.744095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 03:58:52.744095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application conn.query(q) Jun 24 03:58:52.744095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 03:58:52.744095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 03:58:52.744095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 03:58:52.744095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application result.read() Jun 24 03:58:52.744095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 03:58:52.744095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 03:58:52.744095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 03:58:52.744095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 03:58:52.744932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 03:58:52.744932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 03:58:52.744932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 03:58:52.744932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 03:58:52.744932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 03:58:52.744932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 03:58:52.744932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [parameters: {'id': '73006f5405c94c4ab4adb57c2bd472b5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 03:58:52.744932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 03:58:52.744932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:52.744932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:58:52.744932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:52.744932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:52.744932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:52.744932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:52.744932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:52.744932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:52.745766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:52.745766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:52.745766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:52.745766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:52.745766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:52.745766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:52.745766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 24 03:58:52.745766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 24 03:58:52.745766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:52.745766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:52.745766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 24 03:58:52.745766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 24 03:58:52.745766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 03:58:52.745766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 03:58:52.745766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 03:58:52.745766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:52.748543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 242/478] 10.222.0.16 () {60 vars in 1131 bytes} [Fri Jun 24 03:58:52 2022] POST /identity/v3/roles => generated 144 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:58:52.748741 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d27c51-b5ce-4479-bc07-9816c9165a3a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:52.889366 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d27c51-b5ce-4479-bc07-9816c9165a3a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:52.926841 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12e39b-e113-4f00-b766-d04d28710dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:52.927664 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12e39b-e113-4f00-b766-d04d28710dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:52.927960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12e39b-e113-4f00-b766-d04d28710dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:52.928243 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12e39b-e113-4f00-b766-d04d28710dc6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:52.938953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12e39b-e113-4f00-b766-d04d28710dc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:52.947427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-4e12e39b-e113-4f00-b766-d04d28710dc6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:52.947427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:52.947427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:52.947427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:52.947427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:52.947427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:52.947427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:52.947427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:52.947427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:52.947427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:52.947427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:52.947427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:52.947427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:52.947427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:52.947427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:52.947427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:52.949479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:52.949479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:52.949479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:52.949479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:52.949479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:52.949479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:52.949479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:52.949479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:52.949479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:52.949479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:52.949479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:52.949479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:52.949479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:52.949479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:52.949479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:52.949479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:52.950862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:52.950862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:52.950862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:52.950862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:52.950862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:52.950862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:52.950862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:52.950862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:52.950862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:52.950862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:52.950862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 03:58:52.950862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:52.950862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 237/479] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:58:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:52.954689 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4407c-a4c5-4cbb-a7f8-5e245941f1ed None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:53.005161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:53.082007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4407c-a4c5-4cbb-a7f8-5e245941f1ed None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:53.122557 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26844ba3-d5a5-4c72-a40f-9771a1f76ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:53.123291 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26844ba3-d5a5-4c72-a40f-9771a1f76ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:53.123579 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26844ba3-d5a5-4c72-a40f-9771a1f76ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:53.123858 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26844ba3-d5a5-4c72-a40f-9771a1f76ad2 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:53.124660 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26844ba3-d5a5-4c72-a40f-9771a1f76ad2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:53.124975 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26844ba3-d5a5-4c72-a40f-9771a1f76ad2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:53.126354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26844ba3-d5a5-4c72-a40f-9771a1f76ad2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:53.142713 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 243/480] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:58:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:53.813410 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a3d05-59e3-4a4e-bc00-0ab668b36ee2 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:53.813935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a3d05-59e3-4a4e-bc00-0ab668b36ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:53.814011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a3d05-59e3-4a4e-bc00-0ab668b36ee2 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:53.815898 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 238/481] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:53.821810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be7d4b-f3e2-46c8-a19c-65289862a495 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:53.822310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be7d4b-f3e2-46c8-a19c-65289862a495 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:53.822405 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be7d4b-f3e2-46c8-a19c-65289862a495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:53.847418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f4be7d4b-f3e2-46c8-a19c-65289862a495 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:53.848612 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f4be7d4b-f3e2-46c8-a19c-65289862a495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:53.938001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 244/482] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:53.973908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a3d05-59e3-4a4e-bc00-0ab668b36ee2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:54.001230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:54.120542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a3d05-59e3-4a4e-bc00-0ab668b36ee2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:54.182991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931ddcd-97ef-4565-b130-bd2c1755661b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:54.183791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931ddcd-97ef-4565-b130-bd2c1755661b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:54.184082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931ddcd-97ef-4565-b130-bd2c1755661b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:54.184368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931ddcd-97ef-4565-b130-bd2c1755661b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:54.203234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931ddcd-97ef-4565-b130-bd2c1755661b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:54.211163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-a931ddcd-97ef-4565-b130-bd2c1755661b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:58:54.211163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:54.211163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:54.211163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:54.211163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:54.211163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:54.211163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:54.211163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:54.211163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:54.211163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:54.211163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:54.211163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:54.211163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:54.211163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:54.211163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:54.211163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:54.212124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:54.212124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:54.212124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:54.212124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:54.212124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:54.212124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:54.212124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:54.212124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:54.212124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:54.212124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:54.212124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:54.212124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:54.212124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:54.212124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:54.212124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:54.212124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:54.213508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:54.213508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:54.213508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:54.213508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:54.213508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:54.213508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:54.213508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:54.213508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:54.213508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:54.213508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:54.213508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:58:54.213508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:54.213508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 239/483] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:58:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:54.218140 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be7d4b-f3e2-46c8-a19c-65289862a495 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:54.269049 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:54.378927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be7d4b-f3e2-46c8-a19c-65289862a495 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:54.433830 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c3dd1-ed17-4887-ade9-9b898b9f60b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:54.434538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c3dd1-ed17-4887-ade9-9b898b9f60b2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:54.434825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c3dd1-ed17-4887-ade9-9b898b9f60b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:54.435101 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c3dd1-ed17-4887-ade9-9b898b9f60b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:54.435861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c3dd1-ed17-4887-ade9-9b898b9f60b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:54.436173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c3dd1-ed17-4887-ade9-9b898b9f60b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:54.437543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207c3dd1-ed17-4887-ade9-9b898b9f60b2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:54.449893 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 245/484] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 03:58:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:54.455003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931ddcd-97ef-4565-b130-bd2c1755661b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:54.572430 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:54.597973 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931ddcd-97ef-4565-b130-bd2c1755661b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:54.655553 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c66cd-a03d-43a7-8ba2-663677059f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:54.656267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c66cd-a03d-43a7-8ba2-663677059f1c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:54.656550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c66cd-a03d-43a7-8ba2-663677059f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:54.656829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c66cd-a03d-43a7-8ba2-663677059f1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:54.673214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c66cd-a03d-43a7-8ba2-663677059f1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:54.674872 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c66cd-a03d-43a7-8ba2-663677059f1c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:54.695511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-252c66cd-a03d-43a7-8ba2-663677059f1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:54.695511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:54.695511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:54.695511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:54.695511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:54.695511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:54.695511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:54.695511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:54.695511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:54.695511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:54.695511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:54.695511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:54.695511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:58:54.695511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:58:54.695511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:58:54.696404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:58:54.696404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:54.696404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:54.696404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:54.696404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:54.696404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:54.696404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:54.696404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:54.696404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:54.696404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:54.696404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:54.696404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:54.696404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:54.696404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:54.696404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:54.696404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:54.696404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:54.697819 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:58:54.697819 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:58:54.697819 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:54.697819 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:54.697819 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 240/485] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:54.702223 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c3dd1-ed17-4887-ade9-9b898b9f60b2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:54.857050 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:54.870247 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c3dd1-ed17-4887-ade9-9b898b9f60b2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:54.922440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729c9ab-b6f7-4137-b14d-db0e1e7f68ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:54.923011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729c9ab-b6f7-4137-b14d-db0e1e7f68ac None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:54.923291 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729c9ab-b6f7-4137-b14d-db0e1e7f68ac None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:54.923564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729c9ab-b6f7-4137-b14d-db0e1e7f68ac None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:54.924289 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729c9ab-b6f7-4137-b14d-db0e1e7f68ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:54.924673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729c9ab-b6f7-4137-b14d-db0e1e7f68ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:54.926673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729c9ab-b6f7-4137-b14d-db0e1e7f68ac None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:54.944264 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 246/486] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:58:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:54.949137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729c9ab-b6f7-4137-b14d-db0e1e7f68ac None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:54.949249 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c871885-22db-48a8-b91d-952ce70c5717 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:54.949650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c871885-22db-48a8-b91d-952ce70c5717 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:54.949941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c871885-22db-48a8-b91d-952ce70c5717 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:54.951371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 241/487] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:54.957132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fa34c-f006-48cb-b7d1-809592bcd197 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:54.957417 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fa34c-f006-48cb-b7d1-809592bcd197 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:54.957700 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fa34c-f006-48cb-b7d1-809592bcd197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:54.991528 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f99fa34c-f006-48cb-b7d1-809592bcd197 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:54.993191 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f99fa34c-f006-48cb-b7d1-809592bcd197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:55.078926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729c9ab-b6f7-4137-b14d-db0e1e7f68ac None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:55.116153 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff12cac-6222-4201-bce4-b77522e8233b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:55.117091 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff12cac-6222-4201-bce4-b77522e8233b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:55.117380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff12cac-6222-4201-bce4-b77522e8233b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:55.117659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff12cac-6222-4201-bce4-b77522e8233b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:55.124826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 242/488] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:55.126041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff12cac-6222-4201-bce4-b77522e8233b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:55.126240 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea06d6-8554-41ff-9207-aa9bf8a1abd0 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:55.127525 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff12cac-6222-4201-bce4-b77522e8233b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:55.134594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea06d6-8554-41ff-9207-aa9bf8a1abd0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:55.134901 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea06d6-8554-41ff-9207-aa9bf8a1abd0 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:55.136250 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 243/489] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:55 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:55.141525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1155e5f-3bf1-4ca9-bc0f-90c38db6b562 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:55.141817 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1155e5f-3bf1-4ca9-bc0f-90c38db6b562 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:55.142098 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1155e5f-3bf1-4ca9-bc0f-90c38db6b562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:55.145568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-2ff12cac-6222-4201-bce4-b77522e8233b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 03:58:55.145568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:55.145568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:55.145568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:55.145568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:55.145568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:55.145568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:55.145568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:55.145568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:55.145568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:55.145568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:55.145568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:55.145568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:58:55.145568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:58:55.145568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:58:55.145568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:58:55.146492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:55.146492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:55.146492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:58:55.146492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:58:55.146492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:58:55.146492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:58:55.146492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:55.146492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:55.146492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:55.146492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:55.146492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:55.146492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:55.146492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:55.146492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:55.146492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:55.146492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:55.146492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:55.146492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:55.147396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:55.147396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:55.147396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:58:55.147396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:58:55.147396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:58:55.147396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:58:55.147396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:58:55.147396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:58:55.147396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 03:58:55.147396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:55.147396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 247/490] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Jun 24 03:58:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:58:55.152696 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff12cac-6222-4201-bce4-b77522e8233b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:55.167942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e1155e5f-3bf1-4ca9-bc0f-90c38db6b562 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:55.168518 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:55.169736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e1155e5f-3bf1-4ca9-bc0f-90c38db6b562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:55.231282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:55.250636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 244/491] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:58:55.251922 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1155e5f-3bf1-4ca9-bc0f-90c38db6b562 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:55.277029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff12cac-6222-4201-bce4-b77522e8233b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:55.314825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37160fa-cbdf-4988-b371-45d722aada8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:55.315696 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37160fa-cbdf-4988-b371-45d722aada8a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:55.316389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37160fa-cbdf-4988-b371-45d722aada8a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:55.316571 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37160fa-cbdf-4988-b371-45d722aada8a None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:55.317515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37160fa-cbdf-4988-b371-45d722aada8a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:55.317929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37160fa-cbdf-4988-b371-45d722aada8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:55.319630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37160fa-cbdf-4988-b371-45d722aada8a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:55.367750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 248/492] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Jun 24 03:58:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:58:55.369225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37160fa-cbdf-4988-b371-45d722aada8a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:55.383313 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1155e5f-3bf1-4ca9-bc0f-90c38db6b562 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:55.418979 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f08a39-ba4f-429a-a1dc-6bbef2b02950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:55.419776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f08a39-ba4f-429a-a1dc-6bbef2b02950 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:55.420052 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f08a39-ba4f-429a-a1dc-6bbef2b02950 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:55.420336 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f08a39-ba4f-429a-a1dc-6bbef2b02950 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:55.432383 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f08a39-ba4f-429a-a1dc-6bbef2b02950 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:55.440339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e8f08a39-ba4f-429a-a1dc-6bbef2b02950 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:58:55.440339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:55.440339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:55.440339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:55.440339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:55.440339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:55.440339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:55.440339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:55.440339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:55.440339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:55.440339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:55.440339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:55.440339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:55.440339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:55.440339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:55.440339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:55.442125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:55.442125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:55.442125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:55.442125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:55.442125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:55.442125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:55.442125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:55.442125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:55.442125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:55.442125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:55.442125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:55.442125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:55.442125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:55.442125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:55.442125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:55.442125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:55.443120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:55.443120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:55.443120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:55.443120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:55.443120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:55.443120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:55.443120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:55.443120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:55.443120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:55.443120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:55.443120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:58:55.443120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:55.443120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 245/493] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:58:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:55.443870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f08a39-ba4f-429a-a1dc-6bbef2b02950 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:55.449819 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:55.491045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37160fa-cbdf-4988-b371-45d722aada8a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:55.499724 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:55.527605 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a4e14-3bdf-4e46-87ba-b242a0665d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:55.528420 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a4e14-3bdf-4e46-87ba-b242a0665d28 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:55.528709 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a4e14-3bdf-4e46-87ba-b242a0665d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:55.528990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a4e14-3bdf-4e46-87ba-b242a0665d28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:55.540166 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a4e14-3bdf-4e46-87ba-b242a0665d28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:55.547873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-103a4e14-3bdf-4e46-87ba-b242a0665d28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:58:55.547873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:55.547873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:55.547873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:55.547873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:55.547873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:55.547873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:55.547873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:55.547873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:55.547873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:55.547873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:55.547873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:55.547873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:55.547873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:55.547873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:55.547873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:55.548755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:55.548755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:55.548755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:55.548755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:55.548755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:55.548755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:55.548755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:55.548755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:55.548755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:55.548755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:55.548755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:55.548755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:55.548755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:55.548755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:55.548755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:55.548755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:55.550136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:55.550136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:55.550136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:55.550136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:55.550136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:55.550136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:55.550136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:55.550136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:55.550136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:55.550136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:55.550136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:58:55.550136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:55.550136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 249/494] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:58:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:55.551815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a4e14-3bdf-4e46-87ba-b242a0665d28 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:55.561116 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f08a39-ba4f-429a-a1dc-6bbef2b02950 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:55.596992 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4873d-b706-46bb-a983-8569331f2f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:55.597349 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4873d-b706-46bb-a983-8569331f2f2b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:55.597824 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4873d-b706-46bb-a983-8569331f2f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:55.597930 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4873d-b706-46bb-a983-8569331f2f2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:55.660580 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a4e14-3bdf-4e46-87ba-b242a0665d28 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:55.671991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4873d-b706-46bb-a983-8569331f2f2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:55.673468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c4873d-b706-46bb-a983-8569331f2f2b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:55.687047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:55.695744 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad56f2-e0a7-4a8d-af63-6cd6ca2e69a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:55.696409 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad56f2-e0a7-4a8d-af63-6cd6ca2e69a1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:55.696693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad56f2-e0a7-4a8d-af63-6cd6ca2e69a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:55.696965 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad56f2-e0a7-4a8d-af63-6cd6ca2e69a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:55.705310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad56f2-e0a7-4a8d-af63-6cd6ca2e69a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:55.705310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad56f2-e0a7-4a8d-af63-6cd6ca2e69a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:55.705310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad56f2-e0a7-4a8d-af63-6cd6ca2e69a1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:55.712937 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 250/495] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 03:58:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:55.712937 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0683403-38d6-433f-a9bb-65126259ec88 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:55.712937 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0683403-38d6-433f-a9bb-65126259ec88 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:55.712937 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0683403-38d6-433f-a9bb-65126259ec88 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:55.712937 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 251/496] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:55.712937 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0683403-38d6-433f-a9bb-65126259ec88 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:55.737420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:55.787667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 246/497] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:55.788424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4873d-b706-46bb-a983-8569331f2f2b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:55.818588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0683403-38d6-433f-a9bb-65126259ec88 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:55.852497 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a008b22-a07e-4cb6-907c-2cebfcd5d3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:55.853232 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a008b22-a07e-4cb6-907c-2cebfcd5d3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:55.853581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a008b22-a07e-4cb6-907c-2cebfcd5d3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:55.853943 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a008b22-a07e-4cb6-907c-2cebfcd5d3d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:55.854789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a008b22-a07e-4cb6-907c-2cebfcd5d3d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:55.855104 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a008b22-a07e-4cb6-907c-2cebfcd5d3d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:55.856492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a008b22-a07e-4cb6-907c-2cebfcd5d3d5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:55.864917 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 252/498] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 03:58:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:55.866174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef58160-cfb5-46c0-8adb-2bbbc1eecd59 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:55.866489 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef58160-cfb5-46c0-8adb-2bbbc1eecd59 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:55.866771 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef58160-cfb5-46c0-8adb-2bbbc1eecd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:55.883951 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9ef58160-cfb5-46c0-8adb-2bbbc1eecd59 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:55.885137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9ef58160-cfb5-46c0-8adb-2bbbc1eecd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:55.890447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4873d-b706-46bb-a983-8569331f2f2b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:55.925521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67934f-5238-48f6-bee9-ed1640f9b8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:55.926675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67934f-5238-48f6-bee9-ed1640f9b8bf None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:55.926971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67934f-5238-48f6-bee9-ed1640f9b8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:55.927251 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67934f-5238-48f6-bee9-ed1640f9b8bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:55.930944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:55.937464 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67934f-5238-48f6-bee9-ed1640f9b8bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:55.938957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67934f-5238-48f6-bee9-ed1640f9b8bf None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:55.950755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 253/499] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:55.952209 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46171a-5102-4ff0-9ffc-b418a9dd702e None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:55.962728 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46171a-5102-4ff0-9ffc-b418a9dd702e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:55.962728 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46171a-5102-4ff0-9ffc-b418a9dd702e None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:55.962728 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 254/500] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:58:55.962728 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46171a-5102-4ff0-9ffc-b418a9dd702e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:55.963043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-ca67934f-5238-48f6-bee9-ed1640f9b8bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:55.963043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:55.963043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:55.963043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:55.963043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:55.963043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:55.963043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:55.963043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:55.963043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:55.963043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:55.963043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:55.963043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:55.963043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:58:55.963043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:58:55.963043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:58:55.964454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:58:55.964454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:55.964454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:55.964454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:55.964454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:55.964454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:55.964454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:55.964454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:55.964454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:55.964454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:55.964454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:55.964454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:55.964454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:55.964454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:55.964454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:55.964454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:55.964454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:55.965332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:58:55.965332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:58:55.965332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:55.965332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:55.965332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 247/501] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:55.965332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67934f-5238-48f6-bee9-ed1640f9b8bf None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:55.969479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:56.068844 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46171a-5102-4ff0-9ffc-b418a9dd702e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:56.071075 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67934f-5238-48f6-bee9-ed1640f9b8bf None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:56.103472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ed04a-5958-4503-a845-6a7e82ed2270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:56.104257 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ed04a-5958-4503-a845-6a7e82ed2270 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:56.104545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ed04a-5958-4503-a845-6a7e82ed2270 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:56.104821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ed04a-5958-4503-a845-6a7e82ed2270 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:56.105079 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e454a-9292-4040-a8fb-9bbf66474dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:56.105864 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e454a-9292-4040-a8fb-9bbf66474dcd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:56.106142 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e454a-9292-4040-a8fb-9bbf66474dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:56.106451 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e454a-9292-4040-a8fb-9bbf66474dcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:56.116230 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ed04a-5958-4503-a845-6a7e82ed2270 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:56.117007 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e454a-9292-4040-a8fb-9bbf66474dcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:56.117688 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ed04a-5958-4503-a845-6a7e82ed2270 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:56.118477 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e454a-9292-4040-a8fb-9bbf66474dcd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:56.143963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-f17ed04a-5958-4503-a845-6a7e82ed2270 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:56.143963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:56.143963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:56.143963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:56.143963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:56.143963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:56.143963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:56.143963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:56.143963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:56.143963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:56.143963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:56.143963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:56.143963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:58:56.143963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:58:56.143963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:58:56.145069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:58:56.145069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:56.145069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:56.145069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:56.145069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:56.145069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:56.145069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:56.145069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:56.145069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:56.145069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:56.145069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:56.145069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:56.145069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:56.145069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:56.145069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:56.145069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:56.145069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:56.145952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:58:56.145952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:58:56.145952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:56.145952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:56.146162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-6a7e454a-9292-4040-a8fb-9bbf66474dcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:56.146162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:56.146162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:56.146162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:56.146162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:56.146162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:56.146162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:56.146162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:56.146162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:56.146162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:56.146162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:56.146162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:56.146162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:58:56.146162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:58:56.146162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:58:56.147230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:58:56.147230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:56.147230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:56.147230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:56.147230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:56.147230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:56.147230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:56.147230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:56.147230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:56.147230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:56.147230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:56.147230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:56.147230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:56.147230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:56.147230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:56.147230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:56.147230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:56.148068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:58:56.148068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:58:56.148068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:56.148068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:56.148749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 255/502] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:56.148749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e73f7-4648-4890-9d52-fcfcfe744090 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:56.148749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e73f7-4648-4890-9d52-fcfcfe744090 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:56.148749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e73f7-4648-4890-9d52-fcfcfe744090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:56.149593 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 248/503] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:56.150702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e454a-9292-4040-a8fb-9bbf66474dcd None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:56.164615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f10e73f7-4648-4890-9d52-fcfcfe744090 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:58:56.165810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f10e73f7-4648-4890-9d52-fcfcfe744090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:58:56.196740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:56.209721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:56.236876 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 256/504] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:58:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:58:56.238652 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e73f7-4648-4890-9d52-fcfcfe744090 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:56.253939 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e454a-9292-4040-a8fb-9bbf66474dcd None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:56.287939 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3858ca-ac0b-47bd-b486-f640c23a0d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:56.288608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3858ca-ac0b-47bd-b486-f640c23a0d70 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:56.288898 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3858ca-ac0b-47bd-b486-f640c23a0d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:56.289178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3858ca-ac0b-47bd-b486-f640c23a0d70 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:56.289905 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3858ca-ac0b-47bd-b486-f640c23a0d70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:56.290214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3858ca-ac0b-47bd-b486-f640c23a0d70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:56.291641 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3858ca-ac0b-47bd-b486-f640c23a0d70 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:56.303318 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 249/505] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:56.304056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3858ca-ac0b-47bd-b486-f640c23a0d70 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:56.340993 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e73f7-4648-4890-9d52-fcfcfe744090 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:56.376772 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8866aa-ccc2-4725-883c-d41537d40585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:56.377568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8866aa-ccc2-4725-883c-d41537d40585 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:56.378549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8866aa-ccc2-4725-883c-d41537d40585 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:56.378835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8866aa-ccc2-4725-883c-d41537d40585 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:56.389289 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8866aa-ccc2-4725-883c-d41537d40585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:58:56.397495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-9d8866aa-ccc2-4725-883c-d41537d40585 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:58:56.397495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:56.397495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:56.397495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:56.397495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:56.397495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:56.397495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:56.397495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:56.397495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:56.397495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:56.397495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:56.397495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:56.397495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:58:56.397495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:58:56.397495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:58:56.397495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:58:56.398549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:58:56.398549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:58:56.398549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:58:56.398549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:58:56.398549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:56.398549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:56.398549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:56.398549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:56.398549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:56.398549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:56.398549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:56.398549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:56.398549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:56.398549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:56.398549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:56.398549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:56.399391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:56.399391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:56.399391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:56.399391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:56.399391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:58:56.399391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:58:56.399391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:58:56.399391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:58:56.399391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:58:56.399391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:58:56.399391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:58:56.399391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:56.403235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3858ca-ac0b-47bd-b486-f640c23a0d70 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:56.403360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 257/506] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:58:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:56.404176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8866aa-ccc2-4725-883c-d41537d40585 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:56.420015 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:56.430519 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:56.437520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267091e1-d94b-4fa4-a61f-773f4d08623f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:56.439170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267091e1-d94b-4fa4-a61f-773f4d08623f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:56.439170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267091e1-d94b-4fa4-a61f-773f4d08623f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:56.439170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267091e1-d94b-4fa4-a61f-773f4d08623f None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:56.439520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267091e1-d94b-4fa4-a61f-773f4d08623f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:56.439876 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267091e1-d94b-4fa4-a61f-773f4d08623f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:56.441281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267091e1-d94b-4fa4-a61f-773f4d08623f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:56.453635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 250/507] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:56.454456 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267091e1-d94b-4fa4-a61f-773f4d08623f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:56.504542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8866aa-ccc2-4725-883c-d41537d40585 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:56.538174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba210b1-1776-4bac-8ee1-440467afee4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:56.538863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba210b1-1776-4bac-8ee1-440467afee4a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:56.539149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba210b1-1776-4bac-8ee1-440467afee4a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:56.539426 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba210b1-1776-4bac-8ee1-440467afee4a None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:56.540548 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba210b1-1776-4bac-8ee1-440467afee4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:56.540863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba210b1-1776-4bac-8ee1-440467afee4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:56.542253 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba210b1-1776-4bac-8ee1-440467afee4a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:56.553757 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267091e1-d94b-4fa4-a61f-773f4d08623f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:56.554150 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 258/508] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:56.555002 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba210b1-1776-4bac-8ee1-440467afee4a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:56.591337 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91922198-8bc6-4dcb-8f33-f31aada41b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:56.592125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91922198-8bc6-4dcb-8f33-f31aada41b30 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:56.592409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91922198-8bc6-4dcb-8f33-f31aada41b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:56.592684 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91922198-8bc6-4dcb-8f33-f31aada41b30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:56.603274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91922198-8bc6-4dcb-8f33-f31aada41b30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:56.604742 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91922198-8bc6-4dcb-8f33-f31aada41b30 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:56.622702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-91922198-8bc6-4dcb-8f33-f31aada41b30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:56.622702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:56.622702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:56.622702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:56.622702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:56.622702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:56.622702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:56.622702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:56.622702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:56.622702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:56.622702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:56.622702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:56.622702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:58:56.622702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:58:56.622702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:58:56.623737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:58:56.623737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:56.623737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:56.623737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:56.623737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:56.623737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:56.623737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:56.623737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:56.623737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:56.623737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:56.623737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:56.623737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:56.623737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:56.623737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:56.623737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:56.623737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:56.623737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:56.629630 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:58:56.629630 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:58:56.629630 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:56.629630 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:56.629630 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 251/509] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:56.629630 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91922198-8bc6-4dcb-8f33-f31aada41b30 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:56.640298 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:56.656224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba210b1-1776-4bac-8ee1-440467afee4a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:56.657857 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:56.692495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b595759-072a-495d-a3a8-7ae091671b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:56.693296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b595759-072a-495d-a3a8-7ae091671b97 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:56.693582 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b595759-072a-495d-a3a8-7ae091671b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:56.693863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b595759-072a-495d-a3a8-7ae091671b97 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:56.700797 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b595759-072a-495d-a3a8-7ae091671b97 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:56.702244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b595759-072a-495d-a3a8-7ae091671b97 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:56.716181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-3b595759-072a-495d-a3a8-7ae091671b97 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 03:58:56.716181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:56.716181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:56.716181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:56.716181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:56.716181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:56.716181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:56.716181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:56.716181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:56.716181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:56.716181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:56.716181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:56.716181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:58:56.716181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:58:56.716181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:58:56.716181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:58:56.717296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:56.717296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:56.717296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:58:56.717296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:58:56.717296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:58:56.717296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:58:56.717296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:56.717296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:56.717296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:56.717296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:56.717296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:56.717296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:56.717296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:56.717296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:56.717296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:56.717296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:56.717296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:56.717296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:56.718217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:56.718217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:56.718217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:58:56.718217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:58:56.718217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:58:56.718217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:58:56.718217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:58:56.718217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:58:56.718217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 03:58:56.718217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:56.718217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 259/510] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:56.718217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b595759-072a-495d-a3a8-7ae091671b97 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:56.734860 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91922198-8bc6-4dcb-8f33-f31aada41b30 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:56.769981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc26c2b-874c-4c88-88a3-7902105bd9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:56.770674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc26c2b-874c-4c88-88a3-7902105bd9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:56.770961 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc26c2b-874c-4c88-88a3-7902105bd9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:56.771241 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc26c2b-874c-4c88-88a3-7902105bd9b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:56.771978 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc26c2b-874c-4c88-88a3-7902105bd9b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:56.772288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc26c2b-874c-4c88-88a3-7902105bd9b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:56.773665 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc26c2b-874c-4c88-88a3-7902105bd9b2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:56.781395 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 252/511] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 03:58:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:56.782166 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc26c2b-874c-4c88-88a3-7902105bd9b2 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:56.818928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b595759-072a-495d-a3a8-7ae091671b97 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:56.852196 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01501a7-3377-4f0e-a39e-727fe92e8a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:56.852986 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01501a7-3377-4f0e-a39e-727fe92e8a9a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:56.853283 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01501a7-3377-4f0e-a39e-727fe92e8a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:56.853563 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01501a7-3377-4f0e-a39e-727fe92e8a9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:56.860127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01501a7-3377-4f0e-a39e-727fe92e8a9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:56.861592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01501a7-3377-4f0e-a39e-727fe92e8a9a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:56.874088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:56.876697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-c01501a7-3377-4f0e-a39e-727fe92e8a9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:58:56.876697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:56.876697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:56.876697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:56.876697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:56.876697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:56.876697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:56.876697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:56.876697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:56.876697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:56.876697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:56.876697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:56.876697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:58:56.876697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:58:56.876697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:58:56.878161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:58:56.878161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:56.878161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:56.878161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:56.878161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:56.878161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:56.878161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:56.878161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:56.878161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:56.878161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:56.878161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:56.878161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:56.878161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:56.878161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:56.878161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:56.878161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:56.878161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:56.879095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:58:56.879095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:58:56.879095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:58:56.879095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:58:56.879095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:58:56.879095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:58:56.879095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:58:56.879095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:56.879095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 260/512] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:58:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:56.879095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01501a7-3377-4f0e-a39e-727fe92e8a9a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:56.882693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:56.896990 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc26c2b-874c-4c88-88a3-7902105bd9b2 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:56.920369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d447b-962a-4a7f-8721-9aeabb6f0f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:56.921169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d447b-962a-4a7f-8721-9aeabb6f0f27 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:56.921451 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d447b-962a-4a7f-8721-9aeabb6f0f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:56.921727 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d447b-962a-4a7f-8721-9aeabb6f0f27 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:56.929702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d447b-962a-4a7f-8721-9aeabb6f0f27 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:56.931190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d447b-962a-4a7f-8721-9aeabb6f0f27 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:56.948011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-df8d447b-962a-4a7f-8721-9aeabb6f0f27 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 03:58:56.948011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:56.948011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:56.948011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:56.948011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:56.948011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:56.948011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:56.948011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:56.948011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:56.948011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:56.948011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:56.948011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:56.948011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:58:56.948011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:58:56.948011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:58:56.948011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:58:56.949042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:56.949042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:56.949042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:58:56.949042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:58:56.949042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:58:56.949042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:58:56.949042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:56.949042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:56.949042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:56.949042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:56.949042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:56.949042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:56.949042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:56.949042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:56.949042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:56.949042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:56.949042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:56.949042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:56.949915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:56.949915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:56.949915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:58:56.949915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:58:56.949915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:58:56.949915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:58:56.949915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:58:56.949915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:58:56.949915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 03:58:56.949915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:56.949915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 253/513] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:58:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:56.950533 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d447b-962a-4a7f-8721-9aeabb6f0f27 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:56.983891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01501a7-3377-4f0e-a39e-727fe92e8a9a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:57.018385 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db701f03-9107-4515-898f-039ecb447626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:57.019065 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db701f03-9107-4515-898f-039ecb447626 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:57.019349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db701f03-9107-4515-898f-039ecb447626 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:57.019631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db701f03-9107-4515-898f-039ecb447626 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:57.020358 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db701f03-9107-4515-898f-039ecb447626 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:57.020670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db701f03-9107-4515-898f-039ecb447626 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:57.022069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db701f03-9107-4515-898f-039ecb447626 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:57.042941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 261/514] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:57.042941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db701f03-9107-4515-898f-039ecb447626 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:57.055135 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d447b-962a-4a7f-8721-9aeabb6f0f27 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:57.089241 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909237c-20b6-42bb-88c7-248df2ad57e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:57.089841 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909237c-20b6-42bb-88c7-248df2ad57e5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:57.090128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909237c-20b6-42bb-88c7-248df2ad57e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:57.090432 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909237c-20b6-42bb-88c7-248df2ad57e5 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:57.091279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a909237c-20b6-42bb-88c7-248df2ad57e5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:57.091592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a909237c-20b6-42bb-88c7-248df2ad57e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:57.093040 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a909237c-20b6-42bb-88c7-248df2ad57e5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:57.101976 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:57.131936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 254/515] 10.222.0.16 () {56 vars in 1137 bytes} [Fri Jun 24 03:58:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:57.132991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909237c-20b6-42bb-88c7-248df2ad57e5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:57.140354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db701f03-9107-4515-898f-039ecb447626 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:57.142933 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:57.173502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c60cd7-1df1-4fc7-bc12-25eb9ed9cdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:57.174298 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c60cd7-1df1-4fc7-bc12-25eb9ed9cdfa None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:57.175238 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c60cd7-1df1-4fc7-bc12-25eb9ed9cdfa None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:57.175607 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c60cd7-1df1-4fc7-bc12-25eb9ed9cdfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:57.185906 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c60cd7-1df1-4fc7-bc12-25eb9ed9cdfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:57.187410 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c60cd7-1df1-4fc7-bc12-25eb9ed9cdfa None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:57.203952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-61c60cd7-1df1-4fc7-bc12-25eb9ed9cdfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:57.203952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:57.203952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:57.203952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:57.203952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:57.203952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:57.203952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:57.203952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:57.203952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:57.203952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:57.203952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:57.203952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:57.203952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:58:57.203952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:58:57.203952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:58:57.204948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:58:57.204948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:57.204948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:57.204948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:57.204948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:57.204948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:57.204948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:57.204948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:57.204948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:57.204948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:57.204948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:57.204948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:57.204948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:57.204948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:57.204948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:57.204948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:57.204948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:57.205837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:58:57.205837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:58:57.205837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:57.205837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:57.205837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 262/516] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:57.206217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c60cd7-1df1-4fc7-bc12-25eb9ed9cdfa None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:57.240118 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909237c-20b6-42bb-88c7-248df2ad57e5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:57.273953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290cd8b-a152-4fb5-8d19-65856b45dd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:57.274628 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290cd8b-a152-4fb5-8d19-65856b45dd4a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:57.274910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290cd8b-a152-4fb5-8d19-65856b45dd4a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:57.275184 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290cd8b-a152-4fb5-8d19-65856b45dd4a None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:57.275962 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290cd8b-a152-4fb5-8d19-65856b45dd4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:57.276267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290cd8b-a152-4fb5-8d19-65856b45dd4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:57.277699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290cd8b-a152-4fb5-8d19-65856b45dd4a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:57.295989 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 255/517] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:58:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:57.296770 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290cd8b-a152-4fb5-8d19-65856b45dd4a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:57.309215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c60cd7-1df1-4fc7-bc12-25eb9ed9cdfa None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:57.325175 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:57.342360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04ad3e-ec41-4c42-80eb-ac98d8409781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:57.342992 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04ad3e-ec41-4c42-80eb-ac98d8409781 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:57.343276 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04ad3e-ec41-4c42-80eb-ac98d8409781 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:57.343552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf04ad3e-ec41-4c42-80eb-ac98d8409781 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:57.344382 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04ad3e-ec41-4c42-80eb-ac98d8409781 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:57.344689 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04ad3e-ec41-4c42-80eb-ac98d8409781 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:57.346115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf04ad3e-ec41-4c42-80eb-ac98d8409781 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:57.365275 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 263/518] 10.222.0.16 () {56 vars in 1137 bytes} [Fri Jun 24 03:58:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:57.366069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04ad3e-ec41-4c42-80eb-ac98d8409781 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:57.369196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:57.402272 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290cd8b-a152-4fb5-8d19-65856b45dd4a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:57.436424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3678eb8-339b-4fe5-8b42-b18d651f5238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:57.437099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3678eb8-339b-4fe5-8b42-b18d651f5238 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:57.437383 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3678eb8-339b-4fe5-8b42-b18d651f5238 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:57.437663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3678eb8-339b-4fe5-8b42-b18d651f5238 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:57.438387 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3678eb8-339b-4fe5-8b42-b18d651f5238 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:57.439849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3678eb8-339b-4fe5-8b42-b18d651f5238 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:57.465986 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf04ad3e-ec41-4c42-80eb-ac98d8409781 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:57.499064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048db62b-6ae8-43e3-9a70-6e0890de46d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:57.499705 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048db62b-6ae8-43e3-9a70-6e0890de46d3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:57.499985 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048db62b-6ae8-43e3-9a70-6e0890de46d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:57.500261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048db62b-6ae8-43e3-9a70-6e0890de46d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:57.523833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 256/519] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jun 24 03:58:57 2022] POST /identity/v3/users => generated 247 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:58:57.528471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3678eb8-339b-4fe5-8b42-b18d651f5238 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:57.547850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:57.579927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048db62b-6ae8-43e3-9a70-6e0890de46d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:57.582303 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048db62b-6ae8-43e3-9a70-6e0890de46d3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:57.652101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3678eb8-339b-4fe5-8b42-b18d651f5238 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:57.680447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:57.693771 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34e576-4b6e-4923-804a-05667d720edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:57.694623 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34e576-4b6e-4923-804a-05667d720edb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:57.694721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34e576-4b6e-4923-804a-05667d720edb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:57.695002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34e576-4b6e-4923-804a-05667d720edb None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:57.695550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 264/520] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:57.700175 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34e576-4b6e-4923-804a-05667d720edb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:57.700175 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34e576-4b6e-4923-804a-05667d720edb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:57.700349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048db62b-6ae8-43e3-9a70-6e0890de46d3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:57.700425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34e576-4b6e-4923-804a-05667d720edb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:57.712675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 257/521] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:57.713784 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34e576-4b6e-4923-804a-05667d720edb None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:57.784332 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:57.810552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048db62b-6ae8-43e3-9a70-6e0890de46d3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:57.821015 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34e576-4b6e-4923-804a-05667d720edb None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:57.849781 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b6d11-49f3-4dbc-8f18-bf4b22bc975a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:57.850479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b6d11-49f3-4dbc-8f18-bf4b22bc975a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:57.850767 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b6d11-49f3-4dbc-8f18-bf4b22bc975a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:57.851049 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b6d11-49f3-4dbc-8f18-bf4b22bc975a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:57.851886 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b6d11-49f3-4dbc-8f18-bf4b22bc975a None admin] RBAC: Adding query filter params (role.id=3073b5cac26f43089c9ed8c6a5102cff, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=80b7b3ec60db4de19bbf7d08f111f470) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:57.852203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b6d11-49f3-4dbc-8f18-bf4b22bc975a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:57.853650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b6d11-49f3-4dbc-8f18-bf4b22bc975a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:57.856833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e20ef-8677-4995-885c-6af12088d751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:57.857494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e20ef-8677-4995-885c-6af12088d751 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:57.857783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e20ef-8677-4995-885c-6af12088d751 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:57.858066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e20ef-8677-4995-885c-6af12088d751 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:57.866927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 265/522] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:57 2022] GET /identity/v3/role_assignments?user.id=80b7b3ec60db4de19bbf7d08f111f470&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=3073b5cac26f43089c9ed8c6a5102cff => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:57.866927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b6d11-49f3-4dbc-8f18-bf4b22bc975a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:57.931293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:57.951788 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e20ef-8677-4995-885c-6af12088d751 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:57.953536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e20ef-8677-4995-885c-6af12088d751 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:58.013046 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b6d11-49f3-4dbc-8f18-bf4b22bc975a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:58.067615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe25e33-b0cb-46b5-9fa8-0eaf9ba831b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:58.068427 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe25e33-b0cb-46b5-9fa8-0eaf9ba831b8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:58.068712 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe25e33-b0cb-46b5-9fa8-0eaf9ba831b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:58.068991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe25e33-b0cb-46b5-9fa8-0eaf9ba831b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:58.076464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:58.080955 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe25e33-b0cb-46b5-9fa8-0eaf9ba831b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:58.082600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe25e33-b0cb-46b5-9fa8-0eaf9ba831b8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:58.099497 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 258/523] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:58.100361 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e20ef-8677-4995-885c-6af12088d751 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:58.100550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-bfe25e33-b0cb-46b5-9fa8-0eaf9ba831b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:58.100550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:58.100550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:58.100550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:58.100550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:58.100550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:58.100550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:58.100550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:58.100550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:58.100550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:58.100550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:58.100550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:58.100550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:58:58.100550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:58:58.100550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:58:58.101362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:58:58.101362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:58.101362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:58.101362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:58.101362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:58.101362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:58.101362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:58.101362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:58.101362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:58.101362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:58.101362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:58.101362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:58.101362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:58.101362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:58.101362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:58.101362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:58.101362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:58.102296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:58:58.102296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:58:58.102296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:58.102296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:58.102296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 266/524] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:58.107904 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe25e33-b0cb-46b5-9fa8-0eaf9ba831b8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:58.215204 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe25e33-b0cb-46b5-9fa8-0eaf9ba831b8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:58.236050 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e20ef-8677-4995-885c-6af12088d751 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:58.252728 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661baed8-b436-49c7-ac72-c20cf3d762bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:58.253679 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661baed8-b436-49c7-ac72-c20cf3d762bc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:58.254066 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661baed8-b436-49c7-ac72-c20cf3d762bc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:58.254467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661baed8-b436-49c7-ac72-c20cf3d762bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:58.262725 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:58.265991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661baed8-b436-49c7-ac72-c20cf3d762bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:58.267591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661baed8-b436-49c7-ac72-c20cf3d762bc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:58.280684 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f708e-0eca-46fb-9506-09c621a678d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:58.281473 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f708e-0eca-46fb-9506-09c621a678d7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:58.281760 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f708e-0eca-46fb-9506-09c621a678d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:58.282037 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f708e-0eca-46fb-9506-09c621a678d7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:58.287909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-661baed8-b436-49c7-ac72-c20cf3d762bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:58.287909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:58.287909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:58.287909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:58.287909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:58.287909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:58.287909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:58.287909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:58.287909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:58.287909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:58.287909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:58.287909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:58.287909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:58:58.287909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:58:58.287909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:58:58.289638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:58:58.289638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:58.289638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:58.289638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:58.289638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:58.289638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:58.289638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:58.289638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:58.289638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:58.289638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:58.289638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:58.289638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:58.289638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:58.289638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:58.289638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:58.289638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:58.289638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:58.291262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:58:58.291262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:58:58.291262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:58.291262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:58.291642 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f708e-0eca-46fb-9506-09c621a678d7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:58.298411 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f708e-0eca-46fb-9506-09c621a678d7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:58.298634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 267/525] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:58.298634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661baed8-b436-49c7-ac72-c20cf3d762bc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:58.313656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-921f708e-0eca-46fb-9506-09c621a678d7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 03:58:58.313656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:58.313656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:58.313656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:58.313656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:58.313656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:58.313656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:58.313656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:58.313656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:58.313656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:58.313656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:58.313656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:58.313656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:58:58.313656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:58:58.313656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:58:58.313656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:58:58.315699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:58.315699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:58.315699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:58:58.315699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:58:58.315699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:58:58.315699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:58:58.315699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:58.315699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:58.315699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:58.315699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:58.315699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:58.315699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:58.315699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:58.315699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:58.315699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:58.315699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:58.315699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:58.315699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:58.317182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:58.317182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:58.317182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:58:58.317182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:58:58.317182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:58:58.317182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:58:58.317182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:58:58.317182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:58:58.317182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 03:58:58.317182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:58.317182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 259/526] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:58:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:58.317182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f708e-0eca-46fb-9506-09c621a678d7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:58.354714 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:58.431603 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f708e-0eca-46fb-9506-09c621a678d7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:58.436124 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661baed8-b436-49c7-ac72-c20cf3d762bc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:58.473671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15543ec6-4422-49e4-9392-a355168019bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:58.474248 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15543ec6-4422-49e4-9392-a355168019bd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:58.474577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15543ec6-4422-49e4-9392-a355168019bd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:58.474856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15543ec6-4422-49e4-9392-a355168019bd None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:58.475640 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15543ec6-4422-49e4-9392-a355168019bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:58.475894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15543ec6-4422-49e4-9392-a355168019bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:58.477303 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15543ec6-4422-49e4-9392-a355168019bd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:58.481622 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc82e2-ad27-4225-b52b-e6dea5b2c2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:58.482545 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc82e2-ad27-4225-b52b-e6dea5b2c2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:58.483032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc82e2-ad27-4225-b52b-e6dea5b2c2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:58.483588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc82e2-ad27-4225-b52b-e6dea5b2c2c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:58.484509 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc82e2-ad27-4225-b52b-e6dea5b2c2c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:58.485593 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc82e2-ad27-4225-b52b-e6dea5b2c2c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:58.487870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cc82e2-ad27-4225-b52b-e6dea5b2c2c0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:58.490497 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 268/527] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:58.491416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15543ec6-4422-49e4-9392-a355168019bd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:58.504646 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 260/528] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:58:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:58.505875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc82e2-ad27-4225-b52b-e6dea5b2c2c0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:58.513115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:58.606042 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15543ec6-4422-49e4-9392-a355168019bd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:58.607716 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:58.642030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b543c24-e3b8-4796-99b6-a0b0e2130364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:58.642830 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b543c24-e3b8-4796-99b6-a0b0e2130364 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:58.643083 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b543c24-e3b8-4796-99b6-a0b0e2130364 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:58.644028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b543c24-e3b8-4796-99b6-a0b0e2130364 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:58.644841 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b543c24-e3b8-4796-99b6-a0b0e2130364 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:58.645152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b543c24-e3b8-4796-99b6-a0b0e2130364 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:58.646633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b543c24-e3b8-4796-99b6-a0b0e2130364 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:58.656767 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc82e2-ad27-4225-b52b-e6dea5b2c2c0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:58.674857 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 269/529] 10.222.0.16 () {56 vars in 1139 bytes} [Fri Jun 24 03:58:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:58.675998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b543c24-e3b8-4796-99b6-a0b0e2130364 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:58.706457 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd95fc1-f025-43ef-bb9e-98afb7cb846c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:58.707710 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd95fc1-f025-43ef-bb9e-98afb7cb846c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:58.708005 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd95fc1-f025-43ef-bb9e-98afb7cb846c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:58.708547 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd95fc1-f025-43ef-bb9e-98afb7cb846c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:58.718954 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd95fc1-f025-43ef-bb9e-98afb7cb846c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:58.722147 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd95fc1-f025-43ef-bb9e-98afb7cb846c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:58.744586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-0dd95fc1-f025-43ef-bb9e-98afb7cb846c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:58:58.744586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:58.744586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:58.744586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:58.744586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:58.744586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:58.744586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:58.744586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:58.744586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:58.744586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:58.744586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:58.744586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:58.744586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:58:58.744586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:58:58.744586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:58:58.745463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:58:58.745463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:58.745463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:58.745463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:58.745463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:58.745463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:58.745463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:58.745463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:58.745463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:58.745463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:58.745463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:58.745463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:58.745463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:58.745463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:58.745463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:58.745463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:58.745463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:58.746301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:58:58.746301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:58:58.746301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:58:58.746301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:58:58.746301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:58:58.746301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:58:58.746301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:58:58.746301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:58:58.746301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 261/530] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:58:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:58.750416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd95fc1-f025-43ef-bb9e-98afb7cb846c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:58.791773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b543c24-e3b8-4796-99b6-a0b0e2130364 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:58.823511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:58.828860 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fe583-0f71-46fd-b74a-9302f31b6fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:58.829755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fe583-0f71-46fd-b74a-9302f31b6fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:58.830113 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fe583-0f71-46fd-b74a-9302f31b6fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:58.830492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843fe583-0f71-46fd-b74a-9302f31b6fa0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:58.838786 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843fe583-0f71-46fd-b74a-9302f31b6fa0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:58.840352 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843fe583-0f71-46fd-b74a-9302f31b6fa0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:58.857466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-843fe583-0f71-46fd-b74a-9302f31b6fa0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:58:58.857466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:58.857466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:58.857466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:58.857466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:58.857466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:58.857466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:58.857466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:58.857466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:58.857466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:58.857466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:58.857466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:58.857466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:58:58.857466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:58:58.857466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:58:58.858350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:58:58.858350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:58.858350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:58.858350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:58.858350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:58.858350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:58.858350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:58.858350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:58.858350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:58.858350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:58.858350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:58.858350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:58.858350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:58.858350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:58.858350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:58.858350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:58.858350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:58.859367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:58:58.859367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:58:58.859367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:58:58.859367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:58:58.859367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:58:58.859367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:58:58.859367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:58:58.859367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:58.865260 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 270/531] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:58:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:58.866511 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fe583-0f71-46fd-b74a-9302f31b6fa0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:58.870588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd95fc1-f025-43ef-bb9e-98afb7cb846c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:58.898870 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:58.907425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e520e3e-7d81-4ba9-a078-fdebd4659cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:58.913395 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e520e3e-7d81-4ba9-a078-fdebd4659cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:58.913395 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e520e3e-7d81-4ba9-a078-fdebd4659cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:58.913395 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e520e3e-7d81-4ba9-a078-fdebd4659cd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:58.981655 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e520e3e-7d81-4ba9-a078-fdebd4659cd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:58.983442 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e520e3e-7d81-4ba9-a078-fdebd4659cd3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:58.990010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843fe583-0f71-46fd-b74a-9302f31b6fa0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:59.042657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576abe14-785d-4d14-8c67-2ffd2473fb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:59.043139 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576abe14-785d-4d14-8c67-2ffd2473fb57 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:59.043374 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576abe14-785d-4d14-8c67-2ffd2473fb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:59.043569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576abe14-785d-4d14-8c67-2ffd2473fb57 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:59.044391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576abe14-785d-4d14-8c67-2ffd2473fb57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:59.044711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576abe14-785d-4d14-8c67-2ffd2473fb57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:59.046302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576abe14-785d-4d14-8c67-2ffd2473fb57 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:59.054616 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:59.068915 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 271/532] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:58:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:59.070270 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576abe14-785d-4d14-8c67-2ffd2473fb57 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:59.100549 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 262/533] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:58:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:58:59.101581 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e520e3e-7d81-4ba9-a078-fdebd4659cd3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:59.162321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:59.184336 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576abe14-785d-4d14-8c67-2ffd2473fb57 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:59.204765 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e520e3e-7d81-4ba9-a078-fdebd4659cd3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:59.224360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a913a-de0b-46b8-b5af-69520813515a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:59.225267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a913a-de0b-46b8-b5af-69520813515a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:59.225701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a913a-de0b-46b8-b5af-69520813515a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:59.226125 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a913a-de0b-46b8-b5af-69520813515a None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:59.227218 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a913a-de0b-46b8-b5af-69520813515a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:59.227701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a913a-de0b-46b8-b5af-69520813515a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:59.229301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a913a-de0b-46b8-b5af-69520813515a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:59.242139 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041af6a-b2ac-4032-8813-71f3f96bdc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:59.242955 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041af6a-b2ac-4032-8813-71f3f96bdc65 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:59.243338 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041af6a-b2ac-4032-8813-71f3f96bdc65 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:59.243728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041af6a-b2ac-4032-8813-71f3f96bdc65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:59.244187 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 272/534] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:58:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:59.244994 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041af6a-b2ac-4032-8813-71f3f96bdc65 None admin] RBAC: Adding query filter params (role.id=3073b5cac26f43089c9ed8c6a5102cff, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=0fbec7435e0841a5b96946dca814e0c6) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:59.245378 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a913a-de0b-46b8-b5af-69520813515a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:59.245551 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041af6a-b2ac-4032-8813-71f3f96bdc65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:59.247271 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041af6a-b2ac-4032-8813-71f3f96bdc65 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:59.261228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 263/535] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:59 2022] GET /identity/v3/role_assignments?user.id=0fbec7435e0841a5b96946dca814e0c6&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=3073b5cac26f43089c9ed8c6a5102cff => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:59.262387 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041af6a-b2ac-4032-8813-71f3f96bdc65 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:59.294858 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:59.373818 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a913a-de0b-46b8-b5af-69520813515a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:59.375604 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041af6a-b2ac-4032-8813-71f3f96bdc65 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:59.414316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e573b4-e0c4-4ebe-8f08-e76e28ad6e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:59.415247 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106d0d1-8fa2-48bc-89a0-b1f1d5298ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:59.415694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e573b4-e0c4-4ebe-8f08-e76e28ad6e05 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:59.416055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106d0d1-8fa2-48bc-89a0-b1f1d5298ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:59.416376 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e573b4-e0c4-4ebe-8f08-e76e28ad6e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:59.416478 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106d0d1-8fa2-48bc-89a0-b1f1d5298ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:59.416737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106d0d1-8fa2-48bc-89a0-b1f1d5298ee9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:59.417099 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e573b4-e0c4-4ebe-8f08-e76e28ad6e05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:59.417706 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f106d0d1-8fa2-48bc-89a0-b1f1d5298ee9 None admin] RBAC: Adding query filter params (role.id=3073b5cac26f43089c9ed8c6a5102cff, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=61459e15d327405b88a2fdfc1fc0fecd) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:59.418186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f106d0d1-8fa2-48bc-89a0-b1f1d5298ee9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:59.420010 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f106d0d1-8fa2-48bc-89a0-b1f1d5298ee9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:59.427654 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 264/536] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:58:59 2022] GET /identity/v3/role_assignments?user.id=61459e15d327405b88a2fdfc1fc0fecd&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=3073b5cac26f43089c9ed8c6a5102cff => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:59.429186 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:59.434521 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e573b4-e0c4-4ebe-8f08-e76e28ad6e05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:59.434521 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e573b4-e0c4-4ebe-8f08-e76e28ad6e05 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:59.454836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-65e573b4-e0c4-4ebe-8f08-e76e28ad6e05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:59.454836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:59.454836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:59.454836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:59.454836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:59.454836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:59.454836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:59.454836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:59.454836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:59.454836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:59.454836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:59.454836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:59.454836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:58:59.454836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:58:59.454836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:58:59.455959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:58:59.455959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:59.455959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:59.455959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:59.455959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:59.455959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:59.455959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:59.455959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:59.455959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:59.455959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:59.455959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:59.455959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:59.455959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:59.455959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:59.455959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:59.455959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:59.455959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:59.457137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:58:59.457137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:58:59.457137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:58:59.457137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:59.459642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 273/537] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:58:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:58:59.467960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106d0d1-8fa2-48bc-89a0-b1f1d5298ee9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:59.580009 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:58:59.589728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106d0d1-8fa2-48bc-89a0-b1f1d5298ee9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:59.636368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ebe87-9f07-4586-a699-c045ceff6b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:59.637056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ebe87-9f07-4586-a699-c045ceff6b1d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:59.637338 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ebe87-9f07-4586-a699-c045ceff6b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:59.637621 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2ebe87-9f07-4586-a699-c045ceff6b1d None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:59.638608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ebe87-9f07-4586-a699-c045ceff6b1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:58:59.638923 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ebe87-9f07-4586-a699-c045ceff6b1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:59.640680 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2ebe87-9f07-4586-a699-c045ceff6b1d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:59.654627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 265/538] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:58:59.663908 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e573b4-e0c4-4ebe-8f08-e76e28ad6e05 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:58:59.799498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e573b4-e0c4-4ebe-8f08-e76e28ad6e05 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:58:59.833637 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db6bb9-1293-4ecd-93d5-a310b9ab42ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:58:59.834354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db6bb9-1293-4ecd-93d5-a310b9ab42ce None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:58:59.834660 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db6bb9-1293-4ecd-93d5-a310b9ab42ce None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:58:59.834939 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db6bb9-1293-4ecd-93d5-a310b9ab42ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:58:59.842148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db6bb9-1293-4ecd-93d5-a310b9ab42ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:58:59.843626 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db6bb9-1293-4ecd-93d5-a310b9ab42ce None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:58:59.858346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-21db6bb9-1293-4ecd-93d5-a310b9ab42ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:58:59.858346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:58:59.858346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:58:59.858346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:58:59.858346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:58:59.858346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:58:59.858346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:58:59.858346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:58:59.858346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:58:59.858346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:58:59.858346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:58:59.858346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:58:59.858346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:58:59.858346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:58:59.858346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:58:59.859267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:58:59.859267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:58:59.859267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:58:59.859267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:58:59.859267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:58:59.859267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:58:59.859267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:58:59.859267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:58:59.859267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:58:59.859267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:58:59.859267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:58:59.859267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:58:59.859267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:58:59.859267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:58:59.859267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:58:59.859267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:58:59.859267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:58:59.860316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:58:59.860316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:58:59.860316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:58:59.860316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:58:59.860316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:58:59.860316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:58:59.860316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:58:59.860316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:58:59.860316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 274/539] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:58:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:58:59.867266 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ebe87-9f07-4586-a699-c045ceff6b1d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:00.005350 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2ebe87-9f07-4586-a699-c045ceff6b1d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:00.044264 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94663b7-4552-4bcd-8c7c-7e6c88d94c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:00.044644 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94663b7-4552-4bcd-8c7c-7e6c88d94c77 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:00.044931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94663b7-4552-4bcd-8c7c-7e6c88d94c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:00.045211 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94663b7-4552-4bcd-8c7c-7e6c88d94c77 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:00.046073 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94663b7-4552-4bcd-8c7c-7e6c88d94c77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:00.046404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94663b7-4552-4bcd-8c7c-7e6c88d94c77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:00.047245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2596f06-8a1b-4977-a27a-55d9f6658317 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:00.047955 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94663b7-4552-4bcd-8c7c-7e6c88d94c77 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:00.048265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2596f06-8a1b-4977-a27a-55d9f6658317 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:00.048745 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2596f06-8a1b-4977-a27a-55d9f6658317 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:00.054611 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 275/540] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:00.056221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68323cba-7806-49dd-9918-9f8aa7a0c0dc None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:00.056714 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68323cba-7806-49dd-9918-9f8aa7a0c0dc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:00.057214 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68323cba-7806-49dd-9918-9f8aa7a0c0dc None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:00.058735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 276/541] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:00.060373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6293f-b144-4315-98e2-bbc3608ba97d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:00.060879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6293f-b144-4315-98e2-bbc3608ba97d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:00.065782 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 266/542] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:58:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:00.065782 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130beed-a23c-4725-b149-0f4269d7ddb6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:00.065782 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130beed-a23c-4725-b149-0f4269d7ddb6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:00.065782 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130beed-a23c-4725-b149-0f4269d7ddb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:00.066699 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6293f-b144-4315-98e2-bbc3608ba97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:00.084700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-caa6293f-b144-4315-98e2-bbc3608ba97d None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:00.085728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c130beed-a23c-4725-b149-0f4269d7ddb6 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:00.086527 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-caa6293f-b144-4315-98e2-bbc3608ba97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:00.086935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c130beed-a23c-4725-b149-0f4269d7ddb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:00.094968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:00.148023 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:00.177122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 277/543] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:59:00.177122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6293f-b144-4315-98e2-bbc3608ba97d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:00.188408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 267/544] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:59:00.207125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130beed-a23c-4725-b149-0f4269d7ddb6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:00.318587 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6293f-b144-4315-98e2-bbc3608ba97d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:00.331494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130beed-a23c-4725-b149-0f4269d7ddb6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:00.355766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88a65e-47df-4e6a-baa4-443da01f1c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:00.356482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88a65e-47df-4e6a-baa4-443da01f1c7e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:00.356865 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88a65e-47df-4e6a-baa4-443da01f1c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:00.357240 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88a65e-47df-4e6a-baa4-443da01f1c7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:00.358264 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88a65e-47df-4e6a-baa4-443da01f1c7e None admin] RBAC: Adding query filter params (role.id=3073b5cac26f43089c9ed8c6a5102cff, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=fd98b54f587f4b42b4dd0a058b30c684) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:00.358696 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88a65e-47df-4e6a-baa4-443da01f1c7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:00.360218 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88a65e-47df-4e6a-baa4-443da01f1c7e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:00.372054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 278/545] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:59:00 2022] GET /identity/v3/role_assignments?user.id=fd98b54f587f4b42b4dd0a058b30c684&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=3073b5cac26f43089c9ed8c6a5102cff => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:00.378396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88a65e-47df-4e6a-baa4-443da01f1c7e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:00.385933 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359c436-33ec-4ee2-8950-4ba4eb9dbcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:00.386822 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359c436-33ec-4ee2-8950-4ba4eb9dbcc9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:00.387105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359c436-33ec-4ee2-8950-4ba4eb9dbcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:00.387377 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359c436-33ec-4ee2-8950-4ba4eb9dbcc9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:00.411206 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359c436-33ec-4ee2-8950-4ba4eb9dbcc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:00.425546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-c359c436-33ec-4ee2-8950-4ba4eb9dbcc9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:00.425546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:00.425546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:00.425546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:00.425546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:00.425546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:00.425546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:00.425546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:00.425546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:00.425546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:00.425546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:00.425546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:00.425546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:00.425546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:00.425546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:00.425546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:00.426562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:00.426562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:00.426562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:00.426562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:00.426562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:00.426562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:00.426562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:00.426562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:00.426562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:00.426562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:00.426562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:00.426562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:00.426562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:00.426562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:00.426562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:00.426562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:00.427696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:00.427696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:00.427696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:00.427696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:00.427696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:00.427696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:00.427696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:00.427696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:00.427696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:00.427696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:00.427696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:00.427696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:00.427696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 268/546] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:59:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:00.439341 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359c436-33ec-4ee2-8950-4ba4eb9dbcc9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:00.452417 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:00.486234 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:00.577030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88a65e-47df-4e6a-baa4-443da01f1c7e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:00.618351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359c436-33ec-4ee2-8950-4ba4eb9dbcc9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:00.622138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c2cd4-dbb6-4ab4-9094-5b3c2f7b2d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:00.623191 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c2cd4-dbb6-4ab4-9094-5b3c2f7b2d66 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:00.623681 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c2cd4-dbb6-4ab4-9094-5b3c2f7b2d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:00.624151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c2cd4-dbb6-4ab4-9094-5b3c2f7b2d66 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:00.647938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c2cd4-dbb6-4ab4-9094-5b3c2f7b2d66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:00.657463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-9e4c2cd4-dbb6-4ab4-9094-5b3c2f7b2d66 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:00.657463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:00.657463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:00.657463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:00.657463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:00.657463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:00.657463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:00.657463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:00.657463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:00.657463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:00.657463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:00.657463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:00.657463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:00.657463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:00.657463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:00.657463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:00.658452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:00.658452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:00.658452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:00.658452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:00.658452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:00.658452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:00.658452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:00.658452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:00.658452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:00.658452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:00.658452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:00.658452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:00.658452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:00.658452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:00.658452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:00.658452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:00.659296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:00.659296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:00.659296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:00.659296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:00.659296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:00.659296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:00.659296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:00.659296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:00.659296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:00.659296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:00.659296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:00.659296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:00.667198 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 279/547] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:59:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:00.673569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c2cd4-dbb6-4ab4-9094-5b3c2f7b2d66 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:00.698403 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4d255-b1b8-48bc-9837-07d05c8356ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:00.699449 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4d255-b1b8-48bc-9837-07d05c8356ae None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:00.702320 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4d255-b1b8-48bc-9837-07d05c8356ae None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:00.702800 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4d255-b1b8-48bc-9837-07d05c8356ae None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:00.703803 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4d255-b1b8-48bc-9837-07d05c8356ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:00.704256 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4d255-b1b8-48bc-9837-07d05c8356ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:00.707626 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4d255-b1b8-48bc-9837-07d05c8356ae None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:00.719357 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 269/548] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 03:59:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:00.725279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4d255-b1b8-48bc-9837-07d05c8356ae None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:00.799146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:00.845170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4d255-b1b8-48bc-9837-07d05c8356ae None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:00.886897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae298b1e-5ac5-489c-841c-9c15576b882e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:00.887927 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae298b1e-5ac5-489c-841c-9c15576b882e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:00.888351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae298b1e-5ac5-489c-841c-9c15576b882e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:00.889250 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae298b1e-5ac5-489c-841c-9c15576b882e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:00.901923 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae298b1e-5ac5-489c-841c-9c15576b882e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:00.903585 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae298b1e-5ac5-489c-841c-9c15576b882e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:00.924144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-ae298b1e-5ac5-489c-841c-9c15576b882e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:00.924144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:00.924144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:00.924144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:00.924144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:00.924144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:00.924144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:00.924144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:00.924144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:00.924144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:00.924144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:00.924144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:00.924144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:00.924144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:00.924144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:00.926001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:00.926001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:00.926001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:00.926001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:00.926001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:00.926001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:00.926001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:00.926001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:00.926001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:00.926001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:00.926001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:00.926001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:00.926001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:00.926001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:00.926001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:00.926001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:00.926001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:00.927083 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:00.927083 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:00.927083 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:00.927083 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:00.929377 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 270/549] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:00.934906 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae298b1e-5ac5-489c-841c-9c15576b882e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:00.967147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c2cd4-dbb6-4ab4-9094-5b3c2f7b2d66 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:00.984117 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:01.113095 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae298b1e-5ac5-489c-841c-9c15576b882e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:01.132141 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:01.148563 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1cf74-28b9-4816-8db3-bbf082339c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:01.148563 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1cf74-28b9-4816-8db3-bbf082339c2c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:01.148563 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1cf74-28b9-4816-8db3-bbf082339c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:01.148563 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1cf74-28b9-4816-8db3-bbf082339c2c None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:01.149351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1cf74-28b9-4816-8db3-bbf082339c2c None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:01.149351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1cf74-28b9-4816-8db3-bbf082339c2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:01.149351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b1cf74-28b9-4816-8db3-bbf082339c2c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:01.151277 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dcc53b-42a2-45ab-be4e-a04ee200f37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:01.151946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dcc53b-42a2-45ab-be4e-a04ee200f37f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:01.152669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dcc53b-42a2-45ab-be4e-a04ee200f37f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:01.152958 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dcc53b-42a2-45ab-be4e-a04ee200f37f None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:01.153686 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dcc53b-42a2-45ab-be4e-a04ee200f37f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:01.153998 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dcc53b-42a2-45ab-be4e-a04ee200f37f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:01.155646 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dcc53b-42a2-45ab-be4e-a04ee200f37f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:01.161500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 280/550] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 03:59:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 488 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:01.166898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1cf74-28b9-4816-8db3-bbf082339c2c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:01.168990 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 271/551] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:01.174146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dcc53b-42a2-45ab-be4e-a04ee200f37f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:01.332740 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dcc53b-42a2-45ab-be4e-a04ee200f37f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:01.376222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ace0c-44e2-41e1-8e54-f4c6271cbab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:01.377025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ace0c-44e2-41e1-8e54-f4c6271cbab0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:01.377308 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ace0c-44e2-41e1-8e54-f4c6271cbab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:01.378213 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ace0c-44e2-41e1-8e54-f4c6271cbab0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:01.386131 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ace0c-44e2-41e1-8e54-f4c6271cbab0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:01.387928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ace0c-44e2-41e1-8e54-f4c6271cbab0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:01.404620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-fd5ace0c-44e2-41e1-8e54-f4c6271cbab0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 03:59:01.404620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:01.404620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:01.404620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:01.404620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:01.404620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:01.404620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:01.404620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:01.404620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:01.404620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:01.404620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:01.404620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:01.404620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:01.404620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:01.404620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:01.404620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:01.406719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:01.406719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:01.406719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:01.406719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:01.406719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:01.406719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:01.406719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:01.406719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:01.406719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:01.406719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:01.406719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:01.406719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:01.406719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:01.406719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:01.406719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:01.406719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:01.406719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:01.406719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:01.408420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:01.408420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:01.408420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:01.408420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:01.408420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:01.408420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:01.408420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:01.408420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:01.408420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 03:59:01.408420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:01.408420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 272/552] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Jun 24 03:59:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:01.414435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ace0c-44e2-41e1-8e54-f4c6271cbab0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:01.420565 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b1cf74-28b9-4816-8db3-bbf082339c2c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:01.451614 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:01.512329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efd9fe-1094-4dc1-a2d0-30617e0431ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:01.513138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efd9fe-1094-4dc1-a2d0-30617e0431ea None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:01.513431 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efd9fe-1094-4dc1-a2d0-30617e0431ea None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:01.513710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efd9fe-1094-4dc1-a2d0-30617e0431ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:01.524456 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ace0c-44e2-41e1-8e54-f4c6271cbab0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:01.536648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60efd9fe-1094-4dc1-a2d0-30617e0431ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:01.538134 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60efd9fe-1094-4dc1-a2d0-30617e0431ea None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:01.572339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-60efd9fe-1094-4dc1-a2d0-30617e0431ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:01.572339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:01.572339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:01.572339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:01.572339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:01.572339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:01.572339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:01.572339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:01.572339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:01.572339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:01.572339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:01.572339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:01.572339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:01.572339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:01.572339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:01.574650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:01.574650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:01.574650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:01.574650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:01.574650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:01.574650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:01.574650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:01.574650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:01.574650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:01.574650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:01.574650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:01.574650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:01.574650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:01.574650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:01.574650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:01.574650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:01.574650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:01.577706 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:01.577706 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:01.577706 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:01.577706 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:01.577706 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 281/553] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:01.581568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efd9fe-1094-4dc1-a2d0-30617e0431ea None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:01.585931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb9a1a-de3c-44e9-83ca-2723c3d1df92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:01.586868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb9a1a-de3c-44e9-83ca-2723c3d1df92 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:01.587351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb9a1a-de3c-44e9-83ca-2723c3d1df92 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:01.587805 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb9a1a-de3c-44e9-83ca-2723c3d1df92 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:01.588913 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb9a1a-de3c-44e9-83ca-2723c3d1df92 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:01.589493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb9a1a-de3c-44e9-83ca-2723c3d1df92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:01.591181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebb9a1a-de3c-44e9-83ca-2723c3d1df92 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:01.626928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:01.637336 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 273/554] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Jun 24 03:59:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:01.644649 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb9a1a-de3c-44e9-83ca-2723c3d1df92 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:01.746932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efd9fe-1094-4dc1-a2d0-30617e0431ea None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:01.770786 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:01.773818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb9a1a-de3c-44e9-83ca-2723c3d1df92 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:01.782996 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a27ac-588f-4847-86c6-1d6e595bafba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:01.784394 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a27ac-588f-4847-86c6-1d6e595bafba None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:01.784960 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a27ac-588f-4847-86c6-1d6e595bafba None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:01.785555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a27ac-588f-4847-86c6-1d6e595bafba None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:01.786544 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a27ac-588f-4847-86c6-1d6e595bafba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:01.787035 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a27ac-588f-4847-86c6-1d6e595bafba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:01.788588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a27ac-588f-4847-86c6-1d6e595bafba None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:01.801777 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 282/555] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:01.807954 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cad39-5748-439e-8d94-64d88cf0ac85 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:01.808455 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cad39-5748-439e-8d94-64d88cf0ac85 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:01.808916 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cad39-5748-439e-8d94-64d88cf0ac85 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:01.810807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 283/556] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:01.811806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745cad39-5748-439e-8d94-64d88cf0ac85 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:01.817237 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19939df-1e7e-4e94-baf6-a95b3814b48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:01.817922 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19939df-1e7e-4e94-baf6-a95b3814b48d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:01.818200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19939df-1e7e-4e94-baf6-a95b3814b48d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:01.818508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19939df-1e7e-4e94-baf6-a95b3814b48d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:01.833705 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19939df-1e7e-4e94-baf6-a95b3814b48d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:01.835319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19939df-1e7e-4e94-baf6-a95b3814b48d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:01.854362 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-a19939df-1e7e-4e94-baf6-a95b3814b48d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:01.854362 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:01.854362 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:01.854362 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:01.854362 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:01.854362 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:01.854362 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:01.854362 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:01.854362 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:01.854362 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:01.854362 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:01.854362 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:01.854362 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:01.854362 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:01.854362 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:01.855301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:01.855301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:01.855301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:01.855301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:01.855301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:01.855301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:01.855301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:01.855301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:01.855301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:01.855301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:01.855301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:01.855301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:01.855301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:01.855301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:01.855301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:01.855301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:01.855301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:01.856650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:01.856650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:01.856650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:01.856650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:01.856650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 274/557] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:01.857377 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe68374-8fd0-4edf-9556-f094e8827187 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:01.857474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe68374-8fd0-4edf-9556-f094e8827187 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:01.857719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe68374-8fd0-4edf-9556-f094e8827187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:01.875682 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-dfe68374-8fd0-4edf-9556-f094e8827187 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:01.876867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dfe68374-8fd0-4edf-9556-f094e8827187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:01.906329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:01.928731 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745cad39-5748-439e-8d94-64d88cf0ac85 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:01.958758 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 275/558] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:59:01.959690 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe68374-8fd0-4edf-9556-f094e8827187 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:01.965939 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d373491-77c4-4aac-bc02-e65387ef827d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:01.967103 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d373491-77c4-4aac-bc02-e65387ef827d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:01.967625 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d373491-77c4-4aac-bc02-e65387ef827d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:01.968143 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d373491-77c4-4aac-bc02-e65387ef827d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:01.977167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d373491-77c4-4aac-bc02-e65387ef827d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:01.978966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d373491-77c4-4aac-bc02-e65387ef827d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:01.996762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-4d373491-77c4-4aac-bc02-e65387ef827d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 03:59:01.996762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:01.996762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:01.996762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:01.996762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:01.996762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:01.996762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:01.996762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:01.996762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:01.996762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:01.996762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:01.996762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:01.996762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:01.996762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:01.996762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:01.996762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:01.997821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:01.997821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:01.997821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:01.997821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:01.997821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:01.997821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:01.997821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:01.997821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:01.997821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:01.997821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:01.997821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:01.997821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:01.997821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:01.997821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:01.997821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:01.997821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:01.997821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:01.997821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:01.998894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:01.998894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:01.998894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:01.998894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:01.998894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:01.998894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:01.998894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:01.998894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:01.998894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 03:59:01.998894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:01.998894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 284/559] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:59:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:01.999509 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341965e-a263-4418-94f7-96b320505ee0 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:01.999509 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341965e-a263-4418-94f7-96b320505ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:01.999916 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341965e-a263-4418-94f7-96b320505ee0 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:02.001889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 285/560] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:02.002652 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341965e-a263-4418-94f7-96b320505ee0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:02.073085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:02.098210 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe68374-8fd0-4edf-9556-f094e8827187 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:02.123144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341965e-a263-4418-94f7-96b320505ee0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:02.137672 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8d610-c5cd-402b-b6ca-58b552d3e402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:02.138339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8d610-c5cd-402b-b6ca-58b552d3e402 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:02.138657 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8d610-c5cd-402b-b6ca-58b552d3e402 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:02.138938 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c8d610-c5cd-402b-b6ca-58b552d3e402 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:02.139658 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8d610-c5cd-402b-b6ca-58b552d3e402 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:02.139967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8d610-c5cd-402b-b6ca-58b552d3e402 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:02.141522 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c8d610-c5cd-402b-b6ca-58b552d3e402 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:02.153369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 276/561] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:02.154002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8d610-c5cd-402b-b6ca-58b552d3e402 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:02.162615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c976fd-1ce7-46a5-976b-e402527da3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:02.163411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c976fd-1ce7-46a5-976b-e402527da3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:02.163695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c976fd-1ce7-46a5-976b-e402527da3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:02.163971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c976fd-1ce7-46a5-976b-e402527da3f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:02.172056 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:02.175343 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c976fd-1ce7-46a5-976b-e402527da3f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:02.183293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-05c976fd-1ce7-46a5-976b-e402527da3f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:02.183293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:02.183293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:02.183293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:02.183293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:02.183293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:02.183293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:02.183293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:02.183293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:02.183293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:02.183293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:02.183293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:02.183293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:02.183293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:02.183293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:02.183293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:02.191115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:02.191115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:02.191115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:02.191115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:02.191115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:02.191115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:02.191115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:02.191115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:02.191115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:02.191115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:02.191115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:02.191115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:02.191115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:02.191115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:02.191115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:02.191115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:02.191998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:02.191998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:02.191998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:02.191998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:02.191998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:02.191998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:02.191998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:02.191998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:02.191998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:02.191998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:02.191998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:02.191998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:02.191998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 286/562] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:59:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:02.191998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c05923-79d6-452c-a69e-0152d0673b66 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:02.192750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c05923-79d6-452c-a69e-0152d0673b66 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:02.192750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c05923-79d6-452c-a69e-0152d0673b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:02.203396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c0c05923-79d6-452c-a69e-0152d0673b66 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:02.205429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c0c05923-79d6-452c-a69e-0152d0673b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:02.277984 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 287/563] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:59:02.283141 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c8d610-c5cd-402b-b6ca-58b552d3e402 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:02.283375 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c05923-79d6-452c-a69e-0152d0673b66 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:02.325282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fe6d6-85ac-45d3-ac99-c8e0bfaecf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:02.326005 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fe6d6-85ac-45d3-ac99-c8e0bfaecf9f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:02.326297 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fe6d6-85ac-45d3-ac99-c8e0bfaecf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:02.326607 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fe6d6-85ac-45d3-ac99-c8e0bfaecf9f None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:02.327447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2fe6d6-85ac-45d3-ac99-c8e0bfaecf9f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:02.327761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2fe6d6-85ac-45d3-ac99-c8e0bfaecf9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:02.329242 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2fe6d6-85ac-45d3-ac99-c8e0bfaecf9f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:02.332814 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:02.352216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 277/564] 10.222.0.16 () {56 vars in 1135 bytes} [Fri Jun 24 03:59:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:02.352216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fe6d6-85ac-45d3-ac99-c8e0bfaecf9f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:02.410019 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c05923-79d6-452c-a69e-0152d0673b66 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:02.438765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:02.446349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e78b24-c809-4917-a81f-6b52cba043bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:02.447176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e78b24-c809-4917-a81f-6b52cba043bf None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:02.447463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e78b24-c809-4917-a81f-6b52cba043bf None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:02.447766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e78b24-c809-4917-a81f-6b52cba043bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:02.454709 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e78b24-c809-4917-a81f-6b52cba043bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:02.456879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e78b24-c809-4917-a81f-6b52cba043bf None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:02.457780 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fe6d6-85ac-45d3-ac99-c8e0bfaecf9f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:02.470657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-84e78b24-c809-4917-a81f-6b52cba043bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:02.470657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:02.470657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:02.470657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:02.470657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:02.470657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:02.470657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:02.470657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:02.470657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:02.470657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:02.470657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:02.470657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:02.470657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:02.470657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:02.470657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:02.472085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:02.472085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:02.472085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:02.472085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:02.472085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:02.472085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:02.472085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:02.472085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:02.472085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:02.472085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:02.472085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:02.472085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:02.472085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:02.472085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:02.472085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:02.472085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:02.472085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:02.473198 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:02.473198 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:02.473198 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:02.473198 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:02.473198 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:02.473198 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:02.473198 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:02.473198 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:02.473789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 288/565] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:02.479065 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e78b24-c809-4917-a81f-6b52cba043bf None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:02.495881 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266bd846-24fe-48df-8266-5d1ed60b4722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:02.496632 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bd846-24fe-48df-8266-5d1ed60b4722 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:02.496990 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bd846-24fe-48df-8266-5d1ed60b4722 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:02.497340 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bd846-24fe-48df-8266-5d1ed60b4722 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:02.498158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266bd846-24fe-48df-8266-5d1ed60b4722 None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:02.498572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266bd846-24fe-48df-8266-5d1ed60b4722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:02.500013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266bd846-24fe-48df-8266-5d1ed60b4722 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:02.513172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 278/566] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 03:59:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:02.513172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266bd846-24fe-48df-8266-5d1ed60b4722 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:02.579142 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:02.588012 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e78b24-c809-4917-a81f-6b52cba043bf None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:02.622589 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c696fe-361a-4e41-955a-20c7aab0e6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:02.623389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c696fe-361a-4e41-955a-20c7aab0e6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:02.623680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c696fe-361a-4e41-955a-20c7aab0e6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:02.623960 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c696fe-361a-4e41-955a-20c7aab0e6f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:02.628720 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266bd846-24fe-48df-8266-5d1ed60b4722 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:02.634805 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c696fe-361a-4e41-955a-20c7aab0e6f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:02.642551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-24c696fe-361a-4e41-955a-20c7aab0e6f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:02.642551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:02.642551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:02.642551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:02.642551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:02.642551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:02.642551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:02.642551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:02.642551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:02.642551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:02.642551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:02.642551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:02.642551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:02.642551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:02.642551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:02.642551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:02.644272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:02.644272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:02.644272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:02.644272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:02.644272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:02.644272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:02.644272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:02.644272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:02.644272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:02.644272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:02.644272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:02.644272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:02.644272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:02.644272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:02.644272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:02.644272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:02.645174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:02.645174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:02.645174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:02.645174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:02.645174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:02.645174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:02.645174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:02.645174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:02.645174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:02.645174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:02.645174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:02.645174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:02.645174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 289/567] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:59:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:02.645995 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c696fe-361a-4e41-955a-20c7aab0e6f2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:02.674979 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3fdd3-b85f-497a-95e7-6516cdbe298b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:02.676572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3fdd3-b85f-497a-95e7-6516cdbe298b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:02.677179 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3fdd3-b85f-497a-95e7-6516cdbe298b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:02.677825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3fdd3-b85f-497a-95e7-6516cdbe298b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:02.698191 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:02.749678 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3fdd3-b85f-497a-95e7-6516cdbe298b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:02.751179 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade3fdd3-b85f-497a-95e7-6516cdbe298b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:02.751945 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c696fe-361a-4e41-955a-20c7aab0e6f2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:02.785761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045d7f7-809b-4ebf-b743-183ab201ff72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:02.786441 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045d7f7-809b-4ebf-b743-183ab201ff72 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:02.786733 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045d7f7-809b-4ebf-b743-183ab201ff72 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:02.787014 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045d7f7-809b-4ebf-b743-183ab201ff72 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:02.787807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045d7f7-809b-4ebf-b743-183ab201ff72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:02.788119 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045d7f7-809b-4ebf-b743-183ab201ff72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:02.789545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045d7f7-809b-4ebf-b743-183ab201ff72 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:02.801387 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 290/568] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:02.802173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045d7f7-809b-4ebf-b743-183ab201ff72 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:02.810493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:02.855218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 279/569] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:59:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:59:02.856262 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3fdd3-b85f-497a-95e7-6516cdbe298b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:02.899916 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045d7f7-809b-4ebf-b743-183ab201ff72 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:02.910801 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:02.933570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc340e-3054-4f2a-b3e3-40f06fdf0434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:02.934362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc340e-3054-4f2a-b3e3-40f06fdf0434 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:02.934680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc340e-3054-4f2a-b3e3-40f06fdf0434 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:02.934963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc340e-3054-4f2a-b3e3-40f06fdf0434 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:02.945274 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc340e-3054-4f2a-b3e3-40f06fdf0434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:02.947057 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dc340e-3054-4f2a-b3e3-40f06fdf0434 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:02.957229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3fdd3-b85f-497a-95e7-6516cdbe298b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:02.965059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-78dc340e-3054-4f2a-b3e3-40f06fdf0434 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:02.965059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:02.965059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:02.965059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:02.965059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:02.965059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:02.965059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:02.965059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:02.965059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:02.965059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:02.965059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:02.965059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:02.965059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:02.965059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:02.965059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:02.966123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:02.966123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:02.966123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:02.966123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:02.966123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:02.966123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:02.966123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:02.966123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:02.966123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:02.966123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:02.966123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:02.966123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:02.966123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:02.966123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:02.966123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:02.966123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:02.966123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:02.974764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:02.974764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:02.974764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:02.974764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:02.974764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 291/570] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:02.974764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4eceb3-9030-4396-aeca-13f6caeee62d None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:02.974764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4eceb3-9030-4396-aeca-13f6caeee62d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:02.974764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4eceb3-9030-4396-aeca-13f6caeee62d None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:02.974764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 292/571] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:02.975343 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4eceb3-9030-4396-aeca-13f6caeee62d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:02.993348 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c10d0-8db8-4391-ad4d-2f177d91b375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:02.994278 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c10d0-8db8-4391-ad4d-2f177d91b375 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:02.994593 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c10d0-8db8-4391-ad4d-2f177d91b375 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:02.994875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c10d0-8db8-4391-ad4d-2f177d91b375 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:02.995618 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c10d0-8db8-4391-ad4d-2f177d91b375 None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:02.995930 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c10d0-8db8-4391-ad4d-2f177d91b375 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:02.997319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905c10d0-8db8-4391-ad4d-2f177d91b375 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:03.005589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 280/572] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 03:59:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:03.006481 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c10d0-8db8-4391-ad4d-2f177d91b375 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:03.036591 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:03.087488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4eceb3-9030-4396-aeca-13f6caeee62d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:03.120123 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c10d0-8db8-4391-ad4d-2f177d91b375 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:03.120998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb412773-9956-4670-89b9-fa76a29b773f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:03.121887 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb412773-9956-4670-89b9-fa76a29b773f None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:03.122171 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb412773-9956-4670-89b9-fa76a29b773f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:03.122482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb412773-9956-4670-89b9-fa76a29b773f None admin] PATH_INFO: `/v3/projects/68d376986e854a4bb334676572340b4c/users/80b7b3ec60db4de19bbf7d08f111f470/roles/3073b5cac26f43089c9ed8c6a5102cff` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:03.137157 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb412773-9956-4670-89b9-fa76a29b773f None admin] RBAC: Authorizing `identity:create_grant(project_id=68d376986e854a4bb334676572340b4c, user_id=80b7b3ec60db4de19bbf7d08f111f470, role_id=3073b5cac26f43089c9ed8c6a5102cff)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:03.138614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb412773-9956-4670-89b9-fa76a29b773f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:03.153749 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf5d84-8865-4102-ba30-4dc81f95e257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:03.154572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf5d84-8865-4102-ba30-4dc81f95e257 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:03.154859 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf5d84-8865-4102-ba30-4dc81f95e257 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:03.155137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf5d84-8865-4102-ba30-4dc81f95e257 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:03.165538 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf5d84-8865-4102-ba30-4dc81f95e257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:03.167017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf5d84-8865-4102-ba30-4dc81f95e257 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:03.183764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-93bf5d84-8865-4102-ba30-4dc81f95e257 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:03.183764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:03.183764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:03.183764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:03.183764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:03.183764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:03.183764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:03.183764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:03.183764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:03.183764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:03.183764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:03.183764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:03.183764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:03.183764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:03.183764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:03.184816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:03.184816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:03.184816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:03.184816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:03.184816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:03.184816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:03.184816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:03.184816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:03.184816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:03.184816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:03.184816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:03.184816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:03.184816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:03.184816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:03.184816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:03.184816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:03.184816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:03.185818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:03.185818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:03.185818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:03.185818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:03.185818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 281/573] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:03.185818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf5d84-8865-4102-ba30-4dc81f95e257 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:03.214599 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 293/574] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 03:59:02 2022] PUT /identity/v3/projects/68d376986e854a4bb334676572340b4c/users/80b7b3ec60db4de19bbf7d08f111f470/roles/3073b5cac26f43089c9ed8c6a5102cff => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:59:03.215947 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d5425-1233-4a66-81a7-993f4fd4b9e5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:03.216306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d5425-1233-4a66-81a7-993f4fd4b9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:03.216656 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d5425-1233-4a66-81a7-993f4fd4b9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:03.234902 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8d1d5425-1233-4a66-81a7-993f4fd4b9e5 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:03.236136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8d1d5425-1233-4a66-81a7-993f4fd4b9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:03.245597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:03.269608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:03.304536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf5d84-8865-4102-ba30-4dc81f95e257 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:03.313614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 294/575] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:59:03.314830 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d5425-1233-4a66-81a7-993f4fd4b9e5 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:03.355722 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e72625-20a3-4369-9f20-a0d66a5ad237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:03.356456 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e72625-20a3-4369-9f20-a0d66a5ad237 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:03.356693 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e72625-20a3-4369-9f20-a0d66a5ad237 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:03.356973 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e72625-20a3-4369-9f20-a0d66a5ad237 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:03.357709 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e72625-20a3-4369-9f20-a0d66a5ad237 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:03.358020 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e72625-20a3-4369-9f20-a0d66a5ad237 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:03.359612 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e72625-20a3-4369-9f20-a0d66a5ad237 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:03.375323 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 282/576] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:03.375323 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e72625-20a3-4369-9f20-a0d66a5ad237 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:03.430520 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d5425-1233-4a66-81a7-993f4fd4b9e5 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:03.467773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de24e9-fc75-4b40-8317-08f33f9174b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:03.468572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de24e9-fc75-4b40-8317-08f33f9174b0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:03.468863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de24e9-fc75-4b40-8317-08f33f9174b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:03.469146 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de24e9-fc75-4b40-8317-08f33f9174b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:03.479670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de24e9-fc75-4b40-8317-08f33f9174b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:03.481131 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de24e9-fc75-4b40-8317-08f33f9174b0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:03.482935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:03.495302 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e72625-20a3-4369-9f20-a0d66a5ad237 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:03.499788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-b1de24e9-fc75-4b40-8317-08f33f9174b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:03.499788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:03.499788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:03.499788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:03.499788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:03.499788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:03.499788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:03.499788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:03.499788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:03.499788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:03.499788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:03.499788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:03.499788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:03.499788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:03.499788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:03.506268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:03.506268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:03.506268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:03.506268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:03.506268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:03.506268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:03.506268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:03.506268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:03.506268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:03.506268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:03.506268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:03.506268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:03.506268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:03.506268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:03.506268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:03.506268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:03.506268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:03.507392 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:03.507392 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:03.507392 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:03.507392 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:03.507392 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 295/577] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:03.507392 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de24e9-fc75-4b40-8317-08f33f9174b0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:03.531507 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:03.538854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2c0d5-cc3b-4d5d-978f-348f105c67c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:03.539524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2c0d5-cc3b-4d5d-978f-348f105c67c5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:03.539879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2c0d5-cc3b-4d5d-978f-348f105c67c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:03.540168 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c2c0d5-cc3b-4d5d-978f-348f105c67c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:03.540809 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2c0d5-cc3b-4d5d-978f-348f105c67c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:03.541744 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2c0d5-cc3b-4d5d-978f-348f105c67c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:03.542511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c2c0d5-cc3b-4d5d-978f-348f105c67c5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:03.554617 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 283/578] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:03.559748 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2c0d5-cc3b-4d5d-978f-348f105c67c5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:03.612622 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de24e9-fc75-4b40-8317-08f33f9174b0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:03.650140 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e988f0a7-d14e-4a0a-b1f3-a83f13be8657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:03.651361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988f0a7-d14e-4a0a-b1f3-a83f13be8657 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:03.651765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988f0a7-d14e-4a0a-b1f3-a83f13be8657 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:03.652054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988f0a7-d14e-4a0a-b1f3-a83f13be8657 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:03.664205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e988f0a7-d14e-4a0a-b1f3-a83f13be8657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:03.672468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-e988f0a7-d14e-4a0a-b1f3-a83f13be8657 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:03.672468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:03.672468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:03.672468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:03.672468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:03.672468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:03.672468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:03.672468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:03.672468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:03.672468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:03.672468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:03.672468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:03.672468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:03.672468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:03.672468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:03.672468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:03.673586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:03.673586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:03.673586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:03.673586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:03.673586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:03.673586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:03.673586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:03.673586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:03.673586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:03.673586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:03.673586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:03.673586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:03.673586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:03.673586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:03.673586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:03.673586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:03.680532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:03.680532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:03.680532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:03.680532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:03.680532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:03.680532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:03.680532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:03.680532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:03.680532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:03.680532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:03.680532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:03.680532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:03.680532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 296/579] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:59:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:03.681374 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c2c0d5-cc3b-4d5d-978f-348f105c67c5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:03.681460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e988f0a7-d14e-4a0a-b1f3-a83f13be8657 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:03.716617 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39977a09-327f-4e00-b95b-f5464a901b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:03.717510 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39977a09-327f-4e00-b95b-f5464a901b0b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:03.717873 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39977a09-327f-4e00-b95b-f5464a901b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:03.718225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39977a09-327f-4e00-b95b-f5464a901b0b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:03.726420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39977a09-327f-4e00-b95b-f5464a901b0b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:03.727998 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39977a09-327f-4e00-b95b-f5464a901b0b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:03.736702 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:03.743053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-39977a09-327f-4e00-b95b-f5464a901b0b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 03:59:03.743053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:03.743053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:03.743053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:03.743053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:03.743053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:03.743053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:03.743053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:03.743053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:03.743053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:03.743053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:03.743053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:03.743053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:03.743053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:03.743053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:03.743053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:03.744447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:03.744447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:03.744447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:03.744447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:03.744447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:03.744447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:03.744447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:03.744447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:03.744447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:03.744447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:03.744447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:03.744447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:03.744447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:03.744447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:03.744447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:03.744447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:03.744447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:03.744447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:03.750536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:03.750536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:03.750536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:03.750536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:03.750536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:03.750536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:03.750536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:03.750536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:03.750536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 03:59:03.750536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:03.750536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 284/580] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:59:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:03.750536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39977a09-327f-4e00-b95b-f5464a901b0b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:03.793128 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e988f0a7-d14e-4a0a-b1f3-a83f13be8657 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:03.805045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:03.829069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623938ef-c406-47e1-a838-46f1493f13db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:03.829993 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623938ef-c406-47e1-a838-46f1493f13db None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:03.830529 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623938ef-c406-47e1-a838-46f1493f13db None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:03.831014 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623938ef-c406-47e1-a838-46f1493f13db None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:03.831938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623938ef-c406-47e1-a838-46f1493f13db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:03.832430 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623938ef-c406-47e1-a838-46f1493f13db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:03.833999 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623938ef-c406-47e1-a838-46f1493f13db None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:03.846767 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 297/581] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:03.847891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623938ef-c406-47e1-a838-46f1493f13db None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:03.864372 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39977a09-327f-4e00-b95b-f5464a901b0b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:03.899004 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5db960-fa82-4b39-814c-356622b55b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:03.899793 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5db960-fa82-4b39-814c-356622b55b89 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:03.900077 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5db960-fa82-4b39-814c-356622b55b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:03.900407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5db960-fa82-4b39-814c-356622b55b89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:03.907054 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5db960-fa82-4b39-814c-356622b55b89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:03.908519 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5db960-fa82-4b39-814c-356622b55b89 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:03.923530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-fa5db960-fa82-4b39-814c-356622b55b89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:03.923530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:03.923530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:03.923530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:03.923530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:03.923530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:03.923530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:03.923530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:03.923530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:03.923530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:03.923530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:03.923530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:03.923530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:03.923530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:03.923530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:03.925227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:03.925227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:03.925227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:03.925227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:03.925227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:03.925227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:03.925227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:03.925227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:03.925227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:03.925227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:03.925227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:03.925227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:03.925227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:03.925227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:03.925227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:03.925227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:03.925227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:03.926125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:03.926125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:03.926125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:03.926125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:03.926125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:03.926125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:03.926125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:03.926125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:03.926125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 285/582] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:03.927245 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5db960-fa82-4b39-814c-356622b55b89 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:03.966648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623938ef-c406-47e1-a838-46f1493f13db None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:03.982603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:04.005176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1946a-5840-49ba-9797-a31dafec525f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:04.005846 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1946a-5840-49ba-9797-a31dafec525f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:04.006135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1946a-5840-49ba-9797-a31dafec525f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:04.006449 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1946a-5840-49ba-9797-a31dafec525f None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:04.007203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1946a-5840-49ba-9797-a31dafec525f None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:04.007534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1946a-5840-49ba-9797-a31dafec525f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:04.008943 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1946a-5840-49ba-9797-a31dafec525f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:04.016873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 298/583] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 03:59:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:04.017655 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1946a-5840-49ba-9797-a31dafec525f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:04.095314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5db960-fa82-4b39-814c-356622b55b89 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:04.106714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:04.130962 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5258449-dcfa-4d2a-8b1b-8f07540ff8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:04.131631 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5258449-dcfa-4d2a-8b1b-8f07540ff8ad None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:04.131915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5258449-dcfa-4d2a-8b1b-8f07540ff8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:04.132193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5258449-dcfa-4d2a-8b1b-8f07540ff8ad None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:04.133027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5258449-dcfa-4d2a-8b1b-8f07540ff8ad None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:04.133357 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5258449-dcfa-4d2a-8b1b-8f07540ff8ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:04.135077 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5258449-dcfa-4d2a-8b1b-8f07540ff8ad None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:04.152828 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 286/584] 10.222.0.16 () {56 vars in 1137 bytes} [Fri Jun 24 03:59:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:04.154070 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5258449-dcfa-4d2a-8b1b-8f07540ff8ad None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:04.170032 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1946a-5840-49ba-9797-a31dafec525f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:04.206043 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d10f2-0143-4793-8f06-d5a28976a536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:04.207006 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d10f2-0143-4793-8f06-d5a28976a536 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:04.207393 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d10f2-0143-4793-8f06-d5a28976a536 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:04.207761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d10f2-0143-4793-8f06-d5a28976a536 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:04.215372 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d10f2-0143-4793-8f06-d5a28976a536 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:04.216869 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d10f2-0143-4793-8f06-d5a28976a536 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:04.231725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-938d10f2-0143-4793-8f06-d5a28976a536 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 03:59:04.231725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:04.231725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:04.231725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:04.231725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:04.231725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:04.231725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:04.231725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:04.231725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:04.231725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:04.231725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:04.231725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:04.231725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:04.231725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:04.231725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:04.231725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:04.232671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:04.232671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:04.232671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:04.232671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:04.232671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:04.232671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:04.232671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:04.232671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:04.232671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:04.232671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:04.232671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:04.232671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:04.232671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:04.232671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:04.232671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:04.232671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:04.232671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:04.232671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:04.233628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:04.233628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:04.233628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:04.233628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:04.233628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:04.233628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:04.233628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:04.233628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:04.233628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 03:59:04.233628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:04.233628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 299/585] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:59:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:04.234225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d10f2-0143-4793-8f06-d5a28976a536 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:04.266279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5258449-dcfa-4d2a-8b1b-8f07540ff8ad None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:04.273698 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:04.303638 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99565c-a80f-4ef6-b18f-a8f3fb47f31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:04.304536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99565c-a80f-4ef6-b18f-a8f3fb47f31a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:04.305037 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99565c-a80f-4ef6-b18f-a8f3fb47f31a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:04.305484 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99565c-a80f-4ef6-b18f-a8f3fb47f31a None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:04.306442 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99565c-a80f-4ef6-b18f-a8f3fb47f31a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:04.306936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99565c-a80f-4ef6-b18f-a8f3fb47f31a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:04.308381 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99565c-a80f-4ef6-b18f-a8f3fb47f31a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:04.320545 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 287/586] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:04.321390 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99565c-a80f-4ef6-b18f-a8f3fb47f31a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:04.355362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d10f2-0143-4793-8f06-d5a28976a536 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:04.359605 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:04.392379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e8b35-eff4-4f02-b0e9-e2e0d1916984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:04.393721 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e8b35-eff4-4f02-b0e9-e2e0d1916984 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:04.394116 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e8b35-eff4-4f02-b0e9-e2e0d1916984 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:04.394425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e8b35-eff4-4f02-b0e9-e2e0d1916984 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:04.405613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e8b35-eff4-4f02-b0e9-e2e0d1916984 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:04.407196 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281e8b35-eff4-4f02-b0e9-e2e0d1916984 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:04.427898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-281e8b35-eff4-4f02-b0e9-e2e0d1916984 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:04.427898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:04.427898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:04.427898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:04.427898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:04.427898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:04.427898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:04.427898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:04.427898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:04.427898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:04.427898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:04.427898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:04.427898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:04.427898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:04.427898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:04.428997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:04.428997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:04.428997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:04.428997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:04.428997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:04.428997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:04.428997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:04.428997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:04.428997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:04.428997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:04.428997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:04.428997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:04.428997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:04.428997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:04.428997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:04.428997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:04.428997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:04.430094 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:04.430094 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:04.430094 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:04.430094 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:04.435553 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 300/587] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:04.435553 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e8b35-eff4-4f02-b0e9-e2e0d1916984 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:04.446489 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99565c-a80f-4ef6-b18f-a8f3fb47f31a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:04.481924 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0e06f-34bd-4b51-923e-8fa8cf32b515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:04.482822 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0e06f-34bd-4b51-923e-8fa8cf32b515 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:04.483178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0e06f-34bd-4b51-923e-8fa8cf32b515 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:04.483522 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca0e06f-34bd-4b51-923e-8fa8cf32b515 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:04.495209 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0e06f-34bd-4b51-923e-8fa8cf32b515 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:04.496743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca0e06f-34bd-4b51-923e-8fa8cf32b515 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:04.515877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-3ca0e06f-34bd-4b51-923e-8fa8cf32b515 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:04.515877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:04.515877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:04.515877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:04.515877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:04.515877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:04.515877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:04.515877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:04.515877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:04.515877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:04.515877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:04.515877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:04.515877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:04.515877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:04.515877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:04.516959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:04.516959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:04.516959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:04.516959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:04.516959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:04.516959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:04.516959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:04.516959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:04.516959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:04.516959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:04.516959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:04.516959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:04.516959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:04.516959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:04.516959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:04.516959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:04.516959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:04.518795 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:04.518795 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:04.518795 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:04.518795 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:04.524441 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 288/588] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:04.524441 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0e06f-34bd-4b51-923e-8fa8cf32b515 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:04.531428 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:04.544003 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e8b35-eff4-4f02-b0e9-e2e0d1916984 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:04.582055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f75260-ead9-40fb-9f70-059f0a5692b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:04.583113 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f75260-ead9-40fb-9f70-059f0a5692b1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:04.583878 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f75260-ead9-40fb-9f70-059f0a5692b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:04.584227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f75260-ead9-40fb-9f70-059f0a5692b1 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:04.585419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f75260-ead9-40fb-9f70-059f0a5692b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:04.586049 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f75260-ead9-40fb-9f70-059f0a5692b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:04.588347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f75260-ead9-40fb-9f70-059f0a5692b1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:04.605978 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 301/589] 10.222.0.16 () {56 vars in 1137 bytes} [Fri Jun 24 03:59:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:04.606933 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f75260-ead9-40fb-9f70-059f0a5692b1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:04.613805 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:04.634134 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca0e06f-34bd-4b51-923e-8fa8cf32b515 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:04.669070 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9030032-4642-4e84-aee9-d6ac3bd5dedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:04.669861 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9030032-4642-4e84-aee9-d6ac3bd5dedb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:04.670213 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9030032-4642-4e84-aee9-d6ac3bd5dedb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:04.670694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9030032-4642-4e84-aee9-d6ac3bd5dedb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:04.671481 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9030032-4642-4e84-aee9-d6ac3bd5dedb None admin] RBAC: Adding query filter params (role.id=3073b5cac26f43089c9ed8c6a5102cff, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=6d8e1f1e3c7e42cc9fa6f82aaac55129) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:04.671873 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9030032-4642-4e84-aee9-d6ac3bd5dedb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:04.673348 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9030032-4642-4e84-aee9-d6ac3bd5dedb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:04.679510 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 289/590] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:59:04 2022] GET /identity/v3/role_assignments?user.id=6d8e1f1e3c7e42cc9fa6f82aaac55129&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=3073b5cac26f43089c9ed8c6a5102cff => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:04.680511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9030032-4642-4e84-aee9-d6ac3bd5dedb None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:04.730515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f75260-ead9-40fb-9f70-059f0a5692b1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:04.774393 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497daed-3d59-4e53-8564-da4c07e44a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:04.775200 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497daed-3d59-4e53-8564-da4c07e44a3b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:04.775563 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497daed-3d59-4e53-8564-da4c07e44a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:04.775915 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497daed-3d59-4e53-8564-da4c07e44a3b None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:04.776735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d497daed-3d59-4e53-8564-da4c07e44a3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:04.777317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d497daed-3d59-4e53-8564-da4c07e44a3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:04.779330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d497daed-3d59-4e53-8564-da4c07e44a3b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:04.801082 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 302/591] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:04.801082 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497daed-3d59-4e53-8564-da4c07e44a3b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:04.811990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:04.821305 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9030032-4642-4e84-aee9-d6ac3bd5dedb None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:04.868579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c746b-92a9-4077-b6f8-30731cdc77b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:04.869770 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c746b-92a9-4077-b6f8-30731cdc77b6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:04.870124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c746b-92a9-4077-b6f8-30731cdc77b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:04.870520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c746b-92a9-4077-b6f8-30731cdc77b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:04.871667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c746b-92a9-4077-b6f8-30731cdc77b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:04.872054 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c746b-92a9-4077-b6f8-30731cdc77b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:04.873774 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c746b-92a9-4077-b6f8-30731cdc77b6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:04.891235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 290/592] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:04.891235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c746b-92a9-4077-b6f8-30731cdc77b6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:04.900013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:04.935596 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497daed-3d59-4e53-8564-da4c07e44a3b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:04.983364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9a94e-d9d6-4a39-99e5-2e2b472a692f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:04.984522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9a94e-d9d6-4a39-99e5-2e2b472a692f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:04.984957 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9a94e-d9d6-4a39-99e5-2e2b472a692f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:04.985413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9a94e-d9d6-4a39-99e5-2e2b472a692f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:05.001161 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c746b-92a9-4077-b6f8-30731cdc77b6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:05.003575 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9a94e-d9d6-4a39-99e5-2e2b472a692f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:05.006010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9a94e-d9d6-4a39-99e5-2e2b472a692f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:05.029562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-d1a9a94e-d9d6-4a39-99e5-2e2b472a692f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:05.029562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:05.029562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:05.029562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:05.029562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:05.029562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:05.029562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:05.029562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:05.029562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:05.029562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:05.029562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:05.029562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:05.029562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:05.029562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:05.029562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:05.030742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:05.030742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:05.030742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:05.030742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:05.030742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:05.030742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:05.030742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:05.030742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:05.030742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:05.030742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:05.030742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:05.030742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:05.030742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:05.030742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:05.030742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:05.030742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:05.030742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:05.031856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:05.031856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:05.031856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:05.031856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:05.035181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 303/593] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:05.036317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9a94e-d9d6-4a39-99e5-2e2b472a692f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:05.047115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66138c2f-1095-4dec-8bac-790740ab7f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:05.048207 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66138c2f-1095-4dec-8bac-790740ab7f29 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:05.048424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66138c2f-1095-4dec-8bac-790740ab7f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:05.048802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66138c2f-1095-4dec-8bac-790740ab7f29 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:05.057439 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66138c2f-1095-4dec-8bac-790740ab7f29 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:05.059077 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66138c2f-1095-4dec-8bac-790740ab7f29 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:05.074463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-66138c2f-1095-4dec-8bac-790740ab7f29 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 03:59:05.074463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:05.074463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:05.074463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:05.074463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:05.074463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:05.074463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:05.074463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:05.074463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:05.074463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:05.074463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:05.074463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:05.074463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:05.074463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:05.074463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:05.074463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:05.075635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:05.075635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:05.075635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:05.075635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:05.075635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:05.075635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:05.075635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:05.075635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:05.075635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:05.075635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:05.075635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:05.075635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:05.075635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:05.075635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:05.075635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:05.075635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:05.075635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:05.075635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:05.076813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:05.076813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:05.076813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:05.076813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:05.076813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:05.076813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:05.076813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:05.076813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:05.076813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 03:59:05.076813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:05.082889 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 291/594] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:59:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:05.082889 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66138c2f-1095-4dec-8bac-790740ab7f29 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:05.122026 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:05.146925 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:05.178724 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9a94e-d9d6-4a39-99e5-2e2b472a692f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:05.190821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66138c2f-1095-4dec-8bac-790740ab7f29 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:05.217228 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac35a2-2d41-49eb-bfb6-b76a5b521002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:05.218010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac35a2-2d41-49eb-bfb6-b76a5b521002 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:05.218293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac35a2-2d41-49eb-bfb6-b76a5b521002 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:05.218593 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac35a2-2d41-49eb-bfb6-b76a5b521002 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:05.225698 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac35a2-2d41-49eb-bfb6-b76a5b521002 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:05.227210 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac35a2-2d41-49eb-bfb6-b76a5b521002 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:05.237263 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76fadef-56c1-47ce-8e01-ef163d78bde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:05.238265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76fadef-56c1-47ce-8e01-ef163d78bde5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:05.238787 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76fadef-56c1-47ce-8e01-ef163d78bde5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:05.239267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76fadef-56c1-47ce-8e01-ef163d78bde5 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:05.240240 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76fadef-56c1-47ce-8e01-ef163d78bde5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:05.240756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76fadef-56c1-47ce-8e01-ef163d78bde5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:05.242351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76fadef-56c1-47ce-8e01-ef163d78bde5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:05.242974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-b2ac35a2-2d41-49eb-bfb6-b76a5b521002 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:05.242974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:05.242974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:05.242974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:05.242974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:05.242974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:05.242974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:05.242974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:05.242974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:05.242974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:05.242974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:05.242974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:05.242974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:05.242974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:05.242974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:05.244096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:05.244096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:05.244096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:05.244096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:05.244096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:05.244096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:05.244096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:05.244096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:05.244096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:05.244096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:05.244096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:05.244096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:05.244096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:05.244096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:05.244096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:05.244096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:05.244096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:05.245028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:05.245028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:05.245028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:05.245028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:05.245028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:05.245028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:05.245028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:05.245028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:05.249010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 304/595] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:05.249880 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac35a2-2d41-49eb-bfb6-b76a5b521002 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:05.261157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 292/596] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:05.262810 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76fadef-56c1-47ce-8e01-ef163d78bde5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:05.365232 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac35a2-2d41-49eb-bfb6-b76a5b521002 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:05.378116 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76fadef-56c1-47ce-8e01-ef163d78bde5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:05.384522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:05.402387 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:05.403034 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04933205-81bc-4c2f-b0d9-f69700e98eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:05.403874 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04933205-81bc-4c2f-b0d9-f69700e98eee None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:05.404335 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04933205-81bc-4c2f-b0d9-f69700e98eee None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:05.404763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04933205-81bc-4c2f-b0d9-f69700e98eee None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:05.405760 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04933205-81bc-4c2f-b0d9-f69700e98eee None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:05.406076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04933205-81bc-4c2f-b0d9-f69700e98eee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:05.407570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04933205-81bc-4c2f-b0d9-f69700e98eee None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:05.413307 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ea460-27dd-49f8-863b-1ca6f2e57b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:05.413990 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ea460-27dd-49f8-863b-1ca6f2e57b68 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:05.414292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ea460-27dd-49f8-863b-1ca6f2e57b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:05.414629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55ea460-27dd-49f8-863b-1ca6f2e57b68 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:05.415435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ea460-27dd-49f8-863b-1ca6f2e57b68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:05.415748 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ea460-27dd-49f8-863b-1ca6f2e57b68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:05.417184 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55ea460-27dd-49f8-863b-1ca6f2e57b68 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:05.431457 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 293/597] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:05.431457 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ea460-27dd-49f8-863b-1ca6f2e57b68 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:05.431935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 305/598] 10.222.0.16 () {56 vars in 1139 bytes} [Fri Jun 24 03:59:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:05.433604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e6600-1343-4901-b774-148388e72059 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:05.433905 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e6600-1343-4901-b774-148388e72059 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:05.434344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e6600-1343-4901-b774-148388e72059 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:05.435640 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 306/599] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:05.436408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e6600-1343-4901-b774-148388e72059 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:05.543558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e6600-1343-4901-b774-148388e72059 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:05.543738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55ea460-27dd-49f8-863b-1ca6f2e57b68 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:05.577141 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0f06c7-0c30-46e2-af2e-8ef4e6f3bd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:05.577932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f06c7-0c30-46e2-af2e-8ef4e6f3bd4d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:05.578220 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f06c7-0c30-46e2-af2e-8ef4e6f3bd4d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:05.578532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0f06c7-0c30-46e2-af2e-8ef4e6f3bd4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:05.579820 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6b809-2c60-48e9-b7e8-0fc3ea36481d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:05.580735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6b809-2c60-48e9-b7e8-0fc3ea36481d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:05.581229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6b809-2c60-48e9-b7e8-0fc3ea36481d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:05.581511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6b809-2c60-48e9-b7e8-0fc3ea36481d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:05.588376 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6b809-2c60-48e9-b7e8-0fc3ea36481d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:05.588680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f06c7-0c30-46e2-af2e-8ef4e6f3bd4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:05.589816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff6b809-2c60-48e9-b7e8-0fc3ea36481d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:05.590113 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0f06c7-0c30-46e2-af2e-8ef4e6f3bd4d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:05.604660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-0ff6b809-2c60-48e9-b7e8-0fc3ea36481d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:05.604660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:05.604660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:05.604660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:05.604660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:05.604660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:05.604660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:05.604660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:05.604660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:05.604660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:05.604660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:05.604660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:05.604660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:05.604660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:05.604660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:05.605624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:05.605624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:05.605624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:05.605624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:05.605624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:05.605624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:05.605624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:05.605624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:05.605624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:05.605624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:05.605624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:05.605624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:05.605624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:05.605624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:05.605624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:05.605624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:05.605624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:05.606963 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:05.606963 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:05.606963 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:05.606963 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:05.606963 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:05.606963 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:05.606963 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:05.606963 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:05.607761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-4f0f06c7-0c30-46e2-af2e-8ef4e6f3bd4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:05.607761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:05.607761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:05.607761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:05.607761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:05.607761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:05.607761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:05.607761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:05.607761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:05.607761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:05.607761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:05.607761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:05.607761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:05.607761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:05.607761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:05.608585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:05.608585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:05.608585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:05.608585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:05.608585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:05.608585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:05.608585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:05.608585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:05.608585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:05.608585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:05.608585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:05.608585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:05.608585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:05.608585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:05.608585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:05.608585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:05.608585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:05.609474 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:05.609474 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:05.609474 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:05.609474 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:05.610713 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 294/600] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:05.614411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 307/601] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:05.614543 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6b809-2c60-48e9-b7e8-0fc3ea36481d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:05.614658 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f771de-75c8-491f-8d11-5ad90046ddbc None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:05.614658 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f771de-75c8-491f-8d11-5ad90046ddbc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:05.614658 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f771de-75c8-491f-8d11-5ad90046ddbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:05.631086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-52f771de-75c8-491f-8d11-5ad90046ddbc None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:05.632486 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-52f771de-75c8-491f-8d11-5ad90046ddbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:05.655826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:05.661630 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:05.699731 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 308/602] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:59:05.700396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f771de-75c8-491f-8d11-5ad90046ddbc None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:05.727031 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff6b809-2c60-48e9-b7e8-0fc3ea36481d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:05.765154 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb891a8-fdce-4da5-af17-51ecae02e267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:05.766039 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb891a8-fdce-4da5-af17-51ecae02e267 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:05.766323 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb891a8-fdce-4da5-af17-51ecae02e267 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:05.766646 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb891a8-fdce-4da5-af17-51ecae02e267 None admin] PATH_INFO: `/v3/projects/68d376986e854a4bb334676572340b4c/users/0fbec7435e0841a5b96946dca814e0c6/roles/3073b5cac26f43089c9ed8c6a5102cff` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:05.782094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb891a8-fdce-4da5-af17-51ecae02e267 None admin] RBAC: Authorizing `identity:create_grant(project_id=68d376986e854a4bb334676572340b4c, user_id=0fbec7435e0841a5b96946dca814e0c6, role_id=3073b5cac26f43089c9ed8c6a5102cff)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:05.783591 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb891a8-fdce-4da5-af17-51ecae02e267 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:05.806002 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f771de-75c8-491f-8d11-5ad90046ddbc None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:05.840212 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7979302-a89b-4669-9dbf-e3dff8b97de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:05.840893 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7979302-a89b-4669-9dbf-e3dff8b97de1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:05.841176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7979302-a89b-4669-9dbf-e3dff8b97de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:05.841449 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7979302-a89b-4669-9dbf-e3dff8b97de1 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:05.842529 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7979302-a89b-4669-9dbf-e3dff8b97de1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:05.842841 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7979302-a89b-4669-9dbf-e3dff8b97de1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:05.844332 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7979302-a89b-4669-9dbf-e3dff8b97de1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:05.856759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 309/603] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:05.860800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7979302-a89b-4669-9dbf-e3dff8b97de1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:05.877954 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:05.894401 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 295/604] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 03:59:05 2022] PUT /identity/v3/projects/68d376986e854a4bb334676572340b4c/users/0fbec7435e0841a5b96946dca814e0c6/roles/3073b5cac26f43089c9ed8c6a5102cff => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:59:05.894401 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb891a8-fdce-4da5-af17-51ecae02e267 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:05.984891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7979302-a89b-4669-9dbf-e3dff8b97de1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:06.037943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:06.046463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb891a8-fdce-4da5-af17-51ecae02e267 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:06.047893 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f624690-5c0e-40ff-b78c-b61a23e6e402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:06.048767 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f624690-5c0e-40ff-b78c-b61a23e6e402 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:06.049666 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f624690-5c0e-40ff-b78c-b61a23e6e402 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:06.055599 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f624690-5c0e-40ff-b78c-b61a23e6e402 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:06.055599 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f624690-5c0e-40ff-b78c-b61a23e6e402 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:06.055599 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f624690-5c0e-40ff-b78c-b61a23e6e402 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:06.055599 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f624690-5c0e-40ff-b78c-b61a23e6e402 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:06.067212 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 310/605] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:06.067955 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f624690-5c0e-40ff-b78c-b61a23e6e402 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:06.098748 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5fce5-0c9c-47f9-b61f-e1805136f7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:06.100122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5fce5-0c9c-47f9-b61f-e1805136f7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:06.100704 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5fce5-0c9c-47f9-b61f-e1805136f7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:06.101412 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5fce5-0c9c-47f9-b61f-e1805136f7d0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:06.116310 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe5fce5-0c9c-47f9-b61f-e1805136f7d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:06.124326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-ebe5fce5-0c9c-47f9-b61f-e1805136f7d0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:06.124326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:06.124326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:06.124326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:06.124326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:06.124326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:06.124326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:06.124326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:06.124326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:06.124326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:06.124326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:06.124326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:06.124326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:06.124326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:06.124326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:06.124326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:06.125288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:06.125288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:06.125288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:06.125288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:06.125288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:06.125288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:06.125288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:06.125288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:06.125288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:06.125288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:06.125288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:06.125288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:06.125288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:06.125288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:06.125288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:06.125288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:06.126174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:06.126174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:06.126174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:06.126174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:06.126174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:06.126174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:06.126174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:06.126174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:06.126174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:06.126174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:06.126174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:06.126174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:06.130477 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 296/606] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:59:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:06.131334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5fce5-0c9c-47f9-b61f-e1805136f7d0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:06.165245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:06.178566 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f624690-5c0e-40ff-b78c-b61a23e6e402 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:06.212989 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6289b-0b48-418c-9ed6-a1e2b8dc1894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:06.213996 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6289b-0b48-418c-9ed6-a1e2b8dc1894 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:06.214355 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6289b-0b48-418c-9ed6-a1e2b8dc1894 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:06.214734 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6289b-0b48-418c-9ed6-a1e2b8dc1894 None admin] PATH_INFO: `/v3/projects/68d376986e854a4bb334676572340b4c/users/61459e15d327405b88a2fdfc1fc0fecd/roles/3073b5cac26f43089c9ed8c6a5102cff` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:06.230847 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6289b-0b48-418c-9ed6-a1e2b8dc1894 None admin] RBAC: Authorizing `identity:create_grant(project_id=68d376986e854a4bb334676572340b4c, user_id=61459e15d327405b88a2fdfc1fc0fecd, role_id=3073b5cac26f43089c9ed8c6a5102cff)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:06.232433 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6289b-0b48-418c-9ed6-a1e2b8dc1894 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:06.257751 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5fce5-0c9c-47f9-b61f-e1805136f7d0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:06.292912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352002f-9043-47fe-a257-c41486b9d0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:06.294034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352002f-9043-47fe-a257-c41486b9d0ae None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:06.294521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352002f-9043-47fe-a257-c41486b9d0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:06.294963 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352002f-9043-47fe-a257-c41486b9d0ae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:06.302927 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352002f-9043-47fe-a257-c41486b9d0ae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:06.304597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352002f-9043-47fe-a257-c41486b9d0ae None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:06.321969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-6352002f-9043-47fe-a257-c41486b9d0ae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:06.321969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:06.321969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:06.321969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:06.321969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:06.321969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:06.321969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:06.321969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:06.321969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:06.321969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:06.321969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:06.321969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:06.321969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:06.321969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:06.321969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:06.322900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:06.322900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:06.322900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:06.322900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:06.322900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:06.322900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:06.322900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:06.322900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:06.322900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:06.322900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:06.322900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:06.322900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:06.322900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:06.322900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:06.322900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:06.322900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:06.322900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:06.323891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:06.323891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:06.323891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:06.323891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:06.323891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:06.323891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:06.323891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:06.323891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:06.326155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 297/608] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:06.327143 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352002f-9043-47fe-a257-c41486b9d0ae None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:06.331278 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 311/608] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 03:59:06 2022] PUT /identity/v3/projects/68d376986e854a4bb334676572340b4c/users/61459e15d327405b88a2fdfc1fc0fecd/roles/3073b5cac26f43089c9ed8c6a5102cff => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:59:06.336686 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6289b-0b48-418c-9ed6-a1e2b8dc1894 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:06.348902 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:06.442804 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352002f-9043-47fe-a257-c41486b9d0ae None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:06.517320 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6289b-0b48-418c-9ed6-a1e2b8dc1894 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:06.547760 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386451d-c1cb-41ae-ad85-81e3add27cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:06.548458 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386451d-c1cb-41ae-ad85-81e3add27cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:06.548741 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386451d-c1cb-41ae-ad85-81e3add27cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:06.549017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0386451d-c1cb-41ae-ad85-81e3add27cd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:06.551615 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386451d-c1cb-41ae-ad85-81e3add27cd7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:06.551959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386451d-c1cb-41ae-ad85-81e3add27cd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:06.553408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0386451d-c1cb-41ae-ad85-81e3add27cd7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:06.559566 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:06.578544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 298/609] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 03:59:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:06.580153 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386451d-c1cb-41ae-ad85-81e3add27cd7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:06.582793 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ac7f0-047b-4942-8fd0-09e425fb0c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:06.582793 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ac7f0-047b-4942-8fd0-09e425fb0c55 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:06.582793 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ac7f0-047b-4942-8fd0-09e425fb0c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:06.583068 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ac7f0-047b-4942-8fd0-09e425fb0c55 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:06.583875 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ac7f0-047b-4942-8fd0-09e425fb0c55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:06.584900 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ac7f0-047b-4942-8fd0-09e425fb0c55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:06.587708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ac7f0-047b-4942-8fd0-09e425fb0c55 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:06.610419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 312/610] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:06.615016 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ac7f0-047b-4942-8fd0-09e425fb0c55 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:06.701433 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:06.734897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0386451d-c1cb-41ae-ad85-81e3add27cd7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:06.777518 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ac7f0-047b-4942-8fd0-09e425fb0c55 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:06.802204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e043505-404e-4094-817c-bbb2497ed5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:06.804532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e043505-404e-4094-817c-bbb2497ed5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:06.805936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e043505-404e-4094-817c-bbb2497ed5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:06.806384 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e043505-404e-4094-817c-bbb2497ed5d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:06.829523 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e043505-404e-4094-817c-bbb2497ed5d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:06.832620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e043505-404e-4094-817c-bbb2497ed5d8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:06.848960 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c924f-68be-4382-be86-c0e390f9b3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:06.850113 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c924f-68be-4382-be86-c0e390f9b3c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:06.850567 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c924f-68be-4382-be86-c0e390f9b3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:06.851032 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c924f-68be-4382-be86-c0e390f9b3c3 None admin] PATH_INFO: `/v3/projects/68d376986e854a4bb334676572340b4c/users/fd98b54f587f4b42b4dd0a058b30c684/roles/3073b5cac26f43089c9ed8c6a5102cff` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:06.860483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-2e043505-404e-4094-817c-bbb2497ed5d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:06.860483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:06.860483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:06.860483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:06.860483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:06.860483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:06.860483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:06.860483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:06.860483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:06.860483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:06.860483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:06.860483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:06.860483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:06.860483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:06.860483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:06.861407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:06.861407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:06.861407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:06.861407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:06.861407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:06.861407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:06.861407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:06.861407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:06.861407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:06.861407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:06.861407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:06.861407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:06.861407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:06.861407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:06.861407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:06.861407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:06.861407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:06.862314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:06.862314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:06.862314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:06.862314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:06.866502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 299/611] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:06.867819 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42802ec4-3ec2-4e89-b151-665329f0bc22 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:06.868114 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42802ec4-3ec2-4e89-b151-665329f0bc22 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:06.868383 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42802ec4-3ec2-4e89-b151-665329f0bc22 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:06.869995 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c924f-68be-4382-be86-c0e390f9b3c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=68d376986e854a4bb334676572340b4c, user_id=fd98b54f587f4b42b4dd0a058b30c684, role_id=3073b5cac26f43089c9ed8c6a5102cff)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:06.871404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 300/612] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:06.871820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331c924f-68be-4382-be86-c0e390f9b3c3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:06.872128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42802ec4-3ec2-4e89-b151-665329f0bc22 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:06.959180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 313/613] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 03:59:06 2022] PUT /identity/v3/projects/68d376986e854a4bb334676572340b4c/users/fd98b54f587f4b42b4dd0a058b30c684/roles/3073b5cac26f43089c9ed8c6a5102cff => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:59:06.961759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21196c29-a126-4e10-b8e1-ab5ac7ee99b8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:06.962208 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21196c29-a126-4e10-b8e1-ab5ac7ee99b8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:06.966999 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21196c29-a126-4e10-b8e1-ab5ac7ee99b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:07.006222 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-21196c29-a126-4e10-b8e1-ab5ac7ee99b8 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:07.007558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-21196c29-a126-4e10-b8e1-ab5ac7ee99b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:07.011649 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42802ec4-3ec2-4e89-b151-665329f0bc22 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:07.051232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:07.067618 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571011ad-aff2-46f6-9ce1-63979f629ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:07.068416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571011ad-aff2-46f6-9ce1-63979f629ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:07.068708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571011ad-aff2-46f6-9ce1-63979f629ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:07.068986 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571011ad-aff2-46f6-9ce1-63979f629ac2 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:07.069726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571011ad-aff2-46f6-9ce1-63979f629ac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:07.070034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571011ad-aff2-46f6-9ce1-63979f629ac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:07.071654 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571011ad-aff2-46f6-9ce1-63979f629ac2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:07.087575 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:07.096902 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 301/614] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:07.102272 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571011ad-aff2-46f6-9ce1-63979f629ac2 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:07.131814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 314/615] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:59:07.158248 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21196c29-a126-4e10-b8e1-ab5ac7ee99b8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:07.254898 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571011ad-aff2-46f6-9ce1-63979f629ac2 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:07.294775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21196c29-a126-4e10-b8e1-ab5ac7ee99b8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:07.325489 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e18b8e-0d4e-451e-83fa-6d290c71d684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:07.326678 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e18b8e-0d4e-451e-83fa-6d290c71d684 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:07.327201 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e18b8e-0d4e-451e-83fa-6d290c71d684 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:07.327729 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e18b8e-0d4e-451e-83fa-6d290c71d684 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:07.339895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e18b8e-0d4e-451e-83fa-6d290c71d684 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:07.342827 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e18b8e-0d4e-451e-83fa-6d290c71d684 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:07.365046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-b2e18b8e-0d4e-451e-83fa-6d290c71d684 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 03:59:07.365046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:07.365046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:07.365046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:07.365046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:07.365046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:07.365046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:07.365046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:07.365046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:07.365046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:07.365046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:07.365046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:07.365046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:07.365046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:07.365046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:07.365046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:07.366042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:07.366042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:07.366042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:07.366042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:07.366042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:07.366042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:07.366042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:07.366042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:07.366042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:07.366042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:07.366042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:07.366042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:07.366042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:07.366042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:07.366042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:07.366042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:07.366042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:07.366042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:07.367013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:07.367013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:07.367013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:07.367013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:07.367013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:07.367013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:07.367013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:07.367013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:07.367013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 03:59:07.367013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:07.372183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742dfff-1c68-460b-bc03-27142935317b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:07.373257 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742dfff-1c68-460b-bc03-27142935317b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:07.373744 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742dfff-1c68-460b-bc03-27142935317b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:07.374224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742dfff-1c68-460b-bc03-27142935317b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:07.378927 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 302/616] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Jun 24 03:59:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:07.387390 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e18b8e-0d4e-451e-83fa-6d290c71d684 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:07.388535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742dfff-1c68-460b-bc03-27142935317b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:07.398424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-1742dfff-1c68-460b-bc03-27142935317b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:07.398424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:07.398424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:07.398424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:07.398424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:07.398424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:07.398424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:07.398424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:07.398424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:07.398424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:07.398424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:07.398424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:07.398424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:07.398424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:07.398424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:07.398424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:07.399600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:07.399600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:07.399600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:07.399600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:07.399600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:07.399600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:07.399600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:07.399600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:07.399600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:07.399600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:07.399600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:07.399600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:07.399600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:07.399600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:07.399600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:07.399600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:07.400437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:07.400437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:07.400437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:07.400437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:07.400437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:07.400437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:07.400437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:07.400437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:07.400437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:07.400437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:07.400437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:07.400437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:07.400437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 315/617] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:59:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:07.411333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742dfff-1c68-460b-bc03-27142935317b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:07.457352 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:07.495489 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:07.511815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e18b8e-0d4e-451e-83fa-6d290c71d684 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:07.579373 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f3d2d-eec3-42a7-b5af-0e45bd1770c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:07.580053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f3d2d-eec3-42a7-b5af-0e45bd1770c9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:07.580859 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f3d2d-eec3-42a7-b5af-0e45bd1770c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:07.581203 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6f3d2d-eec3-42a7-b5af-0e45bd1770c9 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:07.582007 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6f3d2d-eec3-42a7-b5af-0e45bd1770c9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:07.582317 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6f3d2d-eec3-42a7-b5af-0e45bd1770c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:07.583812 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6f3d2d-eec3-42a7-b5af-0e45bd1770c9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:07.612289 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 303/618] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Jun 24 03:59:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:07.617633 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f3d2d-eec3-42a7-b5af-0e45bd1770c9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:07.633089 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742dfff-1c68-460b-bc03-27142935317b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:07.697145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188c64e-4afe-4116-8b0a-3bcc0d7ba047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:07.699222 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188c64e-4afe-4116-8b0a-3bcc0d7ba047 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:07.700879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188c64e-4afe-4116-8b0a-3bcc0d7ba047 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:07.701433 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188c64e-4afe-4116-8b0a-3bcc0d7ba047 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:07.702584 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8188c64e-4afe-4116-8b0a-3bcc0d7ba047 None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:07.703865 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8188c64e-4afe-4116-8b0a-3bcc0d7ba047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:07.707233 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8188c64e-4afe-4116-8b0a-3bcc0d7ba047 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:07.729762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 316/619] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 03:59:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:07.734454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188c64e-4afe-4116-8b0a-3bcc0d7ba047 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:07.810730 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6f3d2d-eec3-42a7-b5af-0e45bd1770c9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:07.823159 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:07.861676 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188c64e-4afe-4116-8b0a-3bcc0d7ba047 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:07.879568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:07.883757 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa1f492-ad6b-4e5a-aaa4-89b4e58aa457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:07.884423 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1f492-ad6b-4e5a-aaa4-89b4e58aa457 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:07.884770 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1f492-ad6b-4e5a-aaa4-89b4e58aa457 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:07.885049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa1f492-ad6b-4e5a-aaa4-89b4e58aa457 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:07.921378 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1eec3-81e3-4be1-92c1-182c233ca001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:07.922231 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1eec3-81e3-4be1-92c1-182c233ca001 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:07.922549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1eec3-81e3-4be1-92c1-182c233ca001 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:07.922842 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1eec3-81e3-4be1-92c1-182c233ca001 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:07.939205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1eec3-81e3-4be1-92c1-182c233ca001 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:07.941330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1eec3-81e3-4be1-92c1-182c233ca001 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:07.962306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-fad1eec3-81e3-4be1-92c1-182c233ca001 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:07.962306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:07.962306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:07.962306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:07.962306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:07.962306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:07.962306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:07.962306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:07.962306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:07.962306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:07.962306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:07.962306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:07.962306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:07.962306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:07.962306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:07.963347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:07.963347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:07.963347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:07.963347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:07.963347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:07.963347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:07.963347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:07.963347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:07.963347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:07.963347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:07.963347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:07.963347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:07.963347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:07.963347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:07.963347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:07.963347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:07.963347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:07.964277 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:07.964277 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:07.964277 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:07.964277 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:07.968982 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 317/620] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:07.970318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1eec3-81e3-4be1-92c1-182c233ca001 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:08.079244 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1f492-ad6b-4e5a-aaa4-89b4e58aa457 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:08.081035 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa1f492-ad6b-4e5a-aaa4-89b4e58aa457 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:08.101734 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1eec3-81e3-4be1-92c1-182c233ca001 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:08.155871 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2f998-9f1e-4597-bf04-0a9e27b9149b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:08.156464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2f998-9f1e-4597-bf04-0a9e27b9149b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:08.156756 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2f998-9f1e-4597-bf04-0a9e27b9149b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:08.157040 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2f998-9f1e-4597-bf04-0a9e27b9149b None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:08.157768 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2f998-9f1e-4597-bf04-0a9e27b9149b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:08.158083 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2f998-9f1e-4597-bf04-0a9e27b9149b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:08.160185 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa2f998-9f1e-4597-bf04-0a9e27b9149b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:08.190507 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 318/621] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:08.193570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2f998-9f1e-4597-bf04-0a9e27b9149b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:08.199337 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:08.262554 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 304/622] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:59:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:59:08.263979 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecf030-9f00-498e-abc8-abce1fa3e689 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:08.264274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecf030-9f00-498e-abc8-abce1fa3e689 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:08.264557 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecf030-9f00-498e-abc8-abce1fa3e689 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:08.266093 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 305/623] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:08.271363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecf030-9f00-498e-abc8-abce1fa3e689 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:08.280613 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:08.312756 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2f998-9f1e-4597-bf04-0a9e27b9149b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:08.349354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50625f7a-3fb4-4d04-99d8-fb76f664ccc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:08.350159 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50625f7a-3fb4-4d04-99d8-fb76f664ccc1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:08.350560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50625f7a-3fb4-4d04-99d8-fb76f664ccc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:08.350849 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50625f7a-3fb4-4d04-99d8-fb76f664ccc1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:08.358881 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50625f7a-3fb4-4d04-99d8-fb76f664ccc1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:08.360631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50625f7a-3fb4-4d04-99d8-fb76f664ccc1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:08.378167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-50625f7a-3fb4-4d04-99d8-fb76f664ccc1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 03:59:08.378167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:08.378167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:08.378167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:08.378167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:08.378167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:08.378167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:08.378167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:08.378167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:08.378167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:08.378167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:08.378167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:08.378167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:08.378167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:08.378167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:08.378167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:08.379694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:08.379694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:08.379694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:08.379694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:08.379694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:08.379694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:08.379694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:08.379694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:08.379694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:08.379694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:08.379694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:08.379694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:08.379694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:08.379694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:08.379694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:08.379694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:08.379694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:08.379694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:08.380659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:08.380659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:08.380659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:08.380659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:08.380659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:08.380659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:08.380659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:08.380659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:08.380659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 03:59:08.380659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:08.384939 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 319/624] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:59:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:08.385596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecf030-9f00-498e-abc8-abce1fa3e689 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:08.387126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994a736-2caf-44a9-8d38-36a3c4dcb63f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:08.387894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994a736-2caf-44a9-8d38-36a3c4dcb63f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:08.388390 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994a736-2caf-44a9-8d38-36a3c4dcb63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:08.413511 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0994a736-2caf-44a9-8d38-36a3c4dcb63f None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:08.415410 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0994a736-2caf-44a9-8d38-36a3c4dcb63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:08.425250 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33324f86-cfa8-4835-92af-529b545d38d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:08.426056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33324f86-cfa8-4835-92af-529b545d38d8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:08.426347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33324f86-cfa8-4835-92af-529b545d38d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:08.426681 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33324f86-cfa8-4835-92af-529b545d38d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:08.438099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33324f86-cfa8-4835-92af-529b545d38d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:08.439600 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33324f86-cfa8-4835-92af-529b545d38d8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:08.461189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-33324f86-cfa8-4835-92af-529b545d38d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:08.461189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:08.461189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:08.461189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:08.461189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:08.461189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:08.461189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:08.461189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:08.461189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:08.461189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:08.461189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:08.461189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:08.461189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:08.461189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:08.461189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:08.463701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:08.463701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:08.463701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:08.463701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:08.463701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:08.463701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:08.463701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:08.463701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:08.463701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:08.463701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:08.463701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:08.463701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:08.463701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:08.463701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:08.463701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:08.463701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:08.463701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:08.465887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:08.465887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:08.465887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:08.465887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:08.465887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 306/625] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:08.465887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33324f86-cfa8-4835-92af-529b545d38d8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:08.487856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:08.492396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 320/626] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:59:08.494952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994a736-2caf-44a9-8d38-36a3c4dcb63f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:08.550244 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:08.594154 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33324f86-cfa8-4835-92af-529b545d38d8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:08.602732 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994a736-2caf-44a9-8d38-36a3c4dcb63f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:08.630085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758cf2fa-5aec-4b79-8002-53237e8f4df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:08.630805 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cf2fa-5aec-4b79-8002-53237e8f4df6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:08.631094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cf2fa-5aec-4b79-8002-53237e8f4df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:08.631380 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cf2fa-5aec-4b79-8002-53237e8f4df6 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:08.632224 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758cf2fa-5aec-4b79-8002-53237e8f4df6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:08.632539 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758cf2fa-5aec-4b79-8002-53237e8f4df6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:08.633974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758cf2fa-5aec-4b79-8002-53237e8f4df6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:08.642932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47207380-0986-4222-aeac-cadf3bebeb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:08.644603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47207380-0986-4222-aeac-cadf3bebeb9e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:08.644890 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47207380-0986-4222-aeac-cadf3bebeb9e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:08.645173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47207380-0986-4222-aeac-cadf3bebeb9e None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:08.645908 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47207380-0986-4222-aeac-cadf3bebeb9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:08.646221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47207380-0986-4222-aeac-cadf3bebeb9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:08.647630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47207380-0986-4222-aeac-cadf3bebeb9e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:08.659806 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 307/627] 10.222.0.16 () {56 vars in 1135 bytes} [Fri Jun 24 03:59:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:08.660559 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758cf2fa-5aec-4b79-8002-53237e8f4df6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:08.664551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 321/628] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:08.665326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47207380-0986-4222-aeac-cadf3bebeb9e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:08.738792 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:08.791053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758cf2fa-5aec-4b79-8002-53237e8f4df6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:08.809460 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:08.811137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47207380-0986-4222-aeac-cadf3bebeb9e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:08.829407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56821d05-5a42-4937-be28-6971264d71fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:08.830904 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56821d05-5a42-4937-be28-6971264d71fe None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:08.831434 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56821d05-5a42-4937-be28-6971264d71fe None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:08.831735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56821d05-5a42-4937-be28-6971264d71fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:08.843891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56821d05-5a42-4937-be28-6971264d71fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:08.852996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-56821d05-5a42-4937-be28-6971264d71fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:08.852996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:08.852996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:08.852996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:08.852996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:08.852996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:08.852996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:08.852996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:08.852996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:08.852996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:08.852996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:08.852996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:08.852996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:08.852996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:08.852996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:08.852996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:08.854276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:08.854276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:08.854276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:08.854276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:08.854276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:08.854276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:08.854276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:08.854276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:08.854276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:08.854276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:08.854276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:08.854276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:08.854276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:08.854276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:08.854276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:08.854276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:08.855365 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:08.855365 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:08.855365 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:08.855365 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:08.855365 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:08.855365 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:08.855365 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:08.855365 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:08.855365 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:08.855365 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:08.855365 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:08.855365 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:08.860769 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 308/629] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:59:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:08.861625 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56821d05-5a42-4937-be28-6971264d71fe None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:08.862426 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5da44-1600-4822-acdf-9d3860436630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:08.863398 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5da44-1600-4822-acdf-9d3860436630 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:08.863832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5da44-1600-4822-acdf-9d3860436630 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:08.864254 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5da44-1600-4822-acdf-9d3860436630 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:08.881692 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5da44-1600-4822-acdf-9d3860436630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:08.884237 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d5da44-1600-4822-acdf-9d3860436630 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:08.904581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-50d5da44-1600-4822-acdf-9d3860436630 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:08.904581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:08.904581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:08.904581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:08.904581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:08.904581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:08.904581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:08.904581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:08.904581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:08.904581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:08.904581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:08.904581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:08.904581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:08.904581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:08.904581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:08.906749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:08.906749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:08.906749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:08.906749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:08.906749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:08.906749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:08.906749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:08.906749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:08.906749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:08.906749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:08.906749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:08.906749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:08.906749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:08.906749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:08.906749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:08.906749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:08.906749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:08.909967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:08.909967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:08.909967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:08.909967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:08.911110 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 322/630] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:08.911883 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5da44-1600-4822-acdf-9d3860436630 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:08.981678 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56821d05-5a42-4937-be28-6971264d71fe None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:09.015379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5da44-1600-4822-acdf-9d3860436630 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:09.019304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:09.023794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912cd98-cc75-46d3-97a7-11a7a398fe3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:09.024591 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912cd98-cc75-46d3-97a7-11a7a398fe3d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:09.024877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912cd98-cc75-46d3-97a7-11a7a398fe3d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:09.025155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912cd98-cc75-46d3-97a7-11a7a398fe3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:09.033463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912cd98-cc75-46d3-97a7-11a7a398fe3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:09.036240 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912cd98-cc75-46d3-97a7-11a7a398fe3d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:09.054068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-d912cd98-cc75-46d3-97a7-11a7a398fe3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:09.054068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:09.054068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:09.054068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:09.054068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:09.054068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:09.054068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:09.054068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:09.054068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:09.054068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:09.054068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:09.054068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:09.054068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:09.054068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:09.054068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:09.055169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:09.055169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:09.055169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:09.055169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:09.055169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:09.055169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:09.055169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:09.055169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:09.055169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:09.055169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:09.055169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:09.055169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:09.055169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:09.055169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:09.055169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:09.055169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:09.055169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:09.056082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:09.056082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:09.056082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:09.056082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:09.056082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:09.056082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:09.056082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:09.056082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:09.056082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 309/631] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:09.056576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557989ee-ca64-459b-9d3e-b241a97c3760 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:09.057368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557989ee-ca64-459b-9d3e-b241a97c3760 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:09.057368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557989ee-ca64-459b-9d3e-b241a97c3760 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:09.061420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 310/632] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:09.061420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557989ee-ca64-459b-9d3e-b241a97c3760 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:09.068778 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfacb5-f3a4-435a-810e-b2ba3a93d0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:09.069459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfacb5-f3a4-435a-810e-b2ba3a93d0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:09.069750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfacb5-f3a4-435a-810e-b2ba3a93d0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:09.070032 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecfacb5-f3a4-435a-810e-b2ba3a93d0b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:09.070823 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfacb5-f3a4-435a-810e-b2ba3a93d0b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:09.071137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfacb5-f3a4-435a-810e-b2ba3a93d0b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:09.072523 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecfacb5-f3a4-435a-810e-b2ba3a93d0b9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:09.081571 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 323/633] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 03:59:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:09.082568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfacb5-f3a4-435a-810e-b2ba3a93d0b9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:09.105553 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:09.176528 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557989ee-ca64-459b-9d3e-b241a97c3760 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:09.188697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecfacb5-f3a4-435a-810e-b2ba3a93d0b9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:09.213184 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1cb2b-539e-4863-a900-161650325f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:09.213841 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1cb2b-539e-4863-a900-161650325f27 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:09.214129 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1cb2b-539e-4863-a900-161650325f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:09.214430 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1cb2b-539e-4863-a900-161650325f27 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:09.215163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1cb2b-539e-4863-a900-161650325f27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:09.215499 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1cb2b-539e-4863-a900-161650325f27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:09.216870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1cb2b-539e-4863-a900-161650325f27 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:09.224568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f46d5-57e8-43c3-9dc0-7fa921ab0988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:09.225236 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f46d5-57e8-43c3-9dc0-7fa921ab0988 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:09.225525 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f46d5-57e8-43c3-9dc0-7fa921ab0988 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:09.225813 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f46d5-57e8-43c3-9dc0-7fa921ab0988 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:09.226647 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f46d5-57e8-43c3-9dc0-7fa921ab0988 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:09.226967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f46d5-57e8-43c3-9dc0-7fa921ab0988 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:09.228395 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f46d5-57e8-43c3-9dc0-7fa921ab0988 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:09.233780 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 311/634] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:09.233780 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cc34e-830e-4bcf-8bca-5ddd5780eeb8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:09.233780 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cc34e-830e-4bcf-8bca-5ddd5780eeb8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:09.233780 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55cc34e-830e-4bcf-8bca-5ddd5780eeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:09.250929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 324/635] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:09.250929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f46d5-57e8-43c3-9dc0-7fa921ab0988 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:09.252338 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e55cc34e-830e-4bcf-8bca-5ddd5780eeb8 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:09.253731 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e55cc34e-830e-4bcf-8bca-5ddd5780eeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:09.278316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:09.334424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 312/636] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:59:09.334424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cc34e-830e-4bcf-8bca-5ddd5780eeb8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:09.363449 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f46d5-57e8-43c3-9dc0-7fa921ab0988 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:09.365326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:09.397507 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d470d2-99f7-4b98-8109-4c04b58c1941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:09.398295 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d470d2-99f7-4b98-8109-4c04b58c1941 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:09.398606 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d470d2-99f7-4b98-8109-4c04b58c1941 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:09.398884 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d470d2-99f7-4b98-8109-4c04b58c1941 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:09.409234 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d470d2-99f7-4b98-8109-4c04b58c1941 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:09.410702 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d470d2-99f7-4b98-8109-4c04b58c1941 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:09.428512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-e1d470d2-99f7-4b98-8109-4c04b58c1941 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:09.428512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:09.428512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:09.428512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:09.428512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:09.428512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:09.428512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:09.428512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:09.428512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:09.428512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:09.428512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:09.428512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:09.428512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:09.428512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:09.428512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:09.429460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:09.429460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:09.429460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:09.429460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:09.429460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:09.429460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:09.429460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:09.429460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:09.429460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:09.429460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:09.429460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:09.429460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:09.429460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:09.429460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:09.429460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:09.429460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:09.429460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:09.435146 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:09.435146 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:09.435146 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:09.435146 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:09.435146 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 325/637] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:09.435146 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d470d2-99f7-4b98-8109-4c04b58c1941 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:09.437349 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55cc34e-830e-4bcf-8bca-5ddd5780eeb8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:09.470639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235b44c-7b4c-4990-bb35-a62264c82950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:09.471953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235b44c-7b4c-4990-bb35-a62264c82950 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:09.472239 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235b44c-7b4c-4990-bb35-a62264c82950 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:09.472517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235b44c-7b4c-4990-bb35-a62264c82950 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:09.479045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235b44c-7b4c-4990-bb35-a62264c82950 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:09.480492 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235b44c-7b4c-4990-bb35-a62264c82950 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:09.494825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e235b44c-7b4c-4990-bb35-a62264c82950 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:09.494825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:09.494825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:09.494825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:09.494825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:09.494825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:09.494825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:09.494825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:09.494825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:09.494825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:09.494825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:09.494825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:09.494825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:09.494825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:09.494825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:09.495941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:09.495941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:09.495941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:09.495941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:09.495941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:09.495941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:09.495941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:09.495941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:09.495941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:09.495941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:09.495941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:09.495941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:09.495941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:09.495941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:09.495941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:09.495941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:09.495941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:09.497151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:09.497151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:09.497151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:09.497151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:09.497151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:09.497151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:09.497151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:09.497151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:09.497151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 313/638] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:09.497647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907dbb56-a604-4cf2-8929-229051b9479b None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:09.497738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907dbb56-a604-4cf2-8929-229051b9479b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:09.497965 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907dbb56-a604-4cf2-8929-229051b9479b None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:09.499235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 314/639] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:09.499934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907dbb56-a604-4cf2-8929-229051b9479b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:09.516107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:09.540211 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d470d2-99f7-4b98-8109-4c04b58c1941 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:09.573069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d7b7d-9d88-4247-bccd-5443d6d611d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:09.573713 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d7b7d-9d88-4247-bccd-5443d6d611d9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:09.573996 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d7b7d-9d88-4247-bccd-5443d6d611d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:09.574272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d7b7d-9d88-4247-bccd-5443d6d611d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:09.575130 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d7b7d-9d88-4247-bccd-5443d6d611d9 None admin] RBAC: Adding query filter params (role.id=3073b5cac26f43089c9ed8c6a5102cff, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=80b7b3ec60db4de19bbf7d08f111f470) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:09.575443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d7b7d-9d88-4247-bccd-5443d6d611d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:09.576948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356d7b7d-9d88-4247-bccd-5443d6d611d9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:09.583302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 326/640] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:59:09 2022] GET /identity/v3/role_assignments?user.id=80b7b3ec60db4de19bbf7d08f111f470&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=3073b5cac26f43089c9ed8c6a5102cff => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:09.584076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d7b7d-9d88-4247-bccd-5443d6d611d9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:09.590155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:09.610704 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907dbb56-a604-4cf2-8929-229051b9479b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:09.650582 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd53c48-bf3b-45ad-8211-72c4b7e41c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:09.651398 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd53c48-bf3b-45ad-8211-72c4b7e41c82 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:09.651684 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd53c48-bf3b-45ad-8211-72c4b7e41c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:09.651986 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd53c48-bf3b-45ad-8211-72c4b7e41c82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:09.665010 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd53c48-bf3b-45ad-8211-72c4b7e41c82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:09.672988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-3fd53c48-bf3b-45ad-8211-72c4b7e41c82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:09.672988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:09.672988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:09.672988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:09.672988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:09.672988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:09.672988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:09.672988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:09.672988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:09.672988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:09.672988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:09.672988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:09.672988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:09.672988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:09.672988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:09.672988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:09.674181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:09.674181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:09.674181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:09.674181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:09.674181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:09.674181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:09.674181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:09.674181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:09.674181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:09.674181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:09.674181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:09.674181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:09.674181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:09.674181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:09.674181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:09.674181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:09.675281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:09.675281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:09.675281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:09.675281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:09.675281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:09.675281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:09.675281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:09.675281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:09.675281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:09.675281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:09.675281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:09.675281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:09.675281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 315/641] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:59:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:09.675984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd53c48-bf3b-45ad-8211-72c4b7e41c82 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:09.706983 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d7b7d-9d88-4247-bccd-5443d6d611d9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:09.745088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4dca8-fc37-457a-a244-2fa9d9e84059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:09.746347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4dca8-fc37-457a-a244-2fa9d9e84059 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:09.746347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4dca8-fc37-457a-a244-2fa9d9e84059 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:09.746347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4dca8-fc37-457a-a244-2fa9d9e84059 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:09.746995 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd4dca8-fc37-457a-a244-2fa9d9e84059 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:09.747310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd4dca8-fc37-457a-a244-2fa9d9e84059 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:09.748702 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd4dca8-fc37-457a-a244-2fa9d9e84059 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:09.762753 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 327/642] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:09.764109 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93af71-ad64-4d3c-b0be-1eebc08c7be9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:09.764405 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93af71-ad64-4d3c-b0be-1eebc08c7be9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:09.764682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93af71-ad64-4d3c-b0be-1eebc08c7be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:09.770879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:09.783287 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0e93af71-ad64-4d3c-b0be-1eebc08c7be9 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:09.785100 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0e93af71-ad64-4d3c-b0be-1eebc08c7be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:09.798563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd53c48-bf3b-45ad-8211-72c4b7e41c82 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:09.838856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679464b-a2c8-4d10-b726-d70904a42088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:09.840078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679464b-a2c8-4d10-b726-d70904a42088 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:09.840568 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679464b-a2c8-4d10-b726-d70904a42088 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:09.841171 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679464b-a2c8-4d10-b726-d70904a42088 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:09.842565 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679464b-a2c8-4d10-b726-d70904a42088 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:09.843300 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679464b-a2c8-4d10-b726-d70904a42088 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:09.845376 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679464b-a2c8-4d10-b726-d70904a42088 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:09.865076 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 316/643] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:09.865076 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679464b-a2c8-4d10-b726-d70904a42088 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:09.865339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 328/644] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:59:09.867175 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93af71-ad64-4d3c-b0be-1eebc08c7be9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:09.890138 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:09.982168 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93af71-ad64-4d3c-b0be-1eebc08c7be9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:09.985975 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679464b-a2c8-4d10-b726-d70904a42088 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:10.014183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:10.021228 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db6455-23f4-4bd8-9afd-03c79cc52abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:10.022036 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db6455-23f4-4bd8-9afd-03c79cc52abe None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:10.022328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db6455-23f4-4bd8-9afd-03c79cc52abe None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:10.022828 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a6471-d748-4236-a048-4a674288c47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:10.022935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49db6455-23f4-4bd8-9afd-03c79cc52abe None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:10.023010 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a6471-d748-4236-a048-4a674288c47f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:10.023392 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a6471-d748-4236-a048-4a674288c47f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:10.023761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a6471-d748-4236-a048-4a674288c47f None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:10.024509 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a6471-d748-4236-a048-4a674288c47f None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:10.024821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a6471-d748-4236-a048-4a674288c47f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:10.026258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a6471-d748-4236-a048-4a674288c47f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:10.033619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db6455-23f4-4bd8-9afd-03c79cc52abe None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:10.035267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49db6455-23f4-4bd8-9afd-03c79cc52abe None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:10.040417 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 317/645] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 03:59:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:10.040417 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a6471-d748-4236-a048-4a674288c47f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:10.051351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-49db6455-23f4-4bd8-9afd-03c79cc52abe None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 03:59:10.051351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:10.051351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:10.051351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:10.051351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:10.051351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:10.051351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:10.051351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:10.051351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:10.051351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:10.051351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:10.051351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:10.051351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:10.051351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:10.051351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:10.051351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:10.052322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:10.052322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:10.052322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:10.052322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:10.052322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:10.052322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:10.052322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:10.052322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:10.052322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:10.052322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:10.052322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:10.052322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:10.052322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:10.052322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:10.052322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:10.052322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:10.052322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:10.052322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:10.053417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:10.053417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:10.053417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:10.053417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:10.053417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:10.053417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:10.053417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:10.053417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:10.053417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 03:59:10.053417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:10.053417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 329/646] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:59:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:10.053417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db6455-23f4-4bd8-9afd-03c79cc52abe None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:10.125169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:10.162059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a6471-d748-4236-a048-4a674288c47f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:10.175325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49db6455-23f4-4bd8-9afd-03c79cc52abe None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:10.197754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb60f0-6b46-490c-8a95-557a7b582e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:10.198829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb60f0-6b46-490c-8a95-557a7b582e74 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:10.199116 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb60f0-6b46-490c-8a95-557a7b582e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:10.199418 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb60f0-6b46-490c-8a95-557a7b582e74 None admin] PATH_INFO: `/v3/projects/68d376986e854a4bb334676572340b4c/users/6d8e1f1e3c7e42cc9fa6f82aaac55129/roles/3073b5cac26f43089c9ed8c6a5102cff` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:10.211696 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ccadb-10a4-465f-bcf8-b0176c3606ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:10.212551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ccadb-10a4-465f-bcf8-b0176c3606ca None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:10.212843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ccadb-10a4-465f-bcf8-b0176c3606ca None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:10.213202 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ccadb-10a4-465f-bcf8-b0176c3606ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:10.215126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb60f0-6b46-490c-8a95-557a7b582e74 None admin] RBAC: Authorizing `identity:create_grant(project_id=68d376986e854a4bb334676572340b4c, user_id=6d8e1f1e3c7e42cc9fa6f82aaac55129, role_id=3073b5cac26f43089c9ed8c6a5102cff)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:10.216574 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeb60f0-6b46-490c-8a95-557a7b582e74 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:10.223676 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ccadb-10a4-465f-bcf8-b0176c3606ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:10.231406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-231ccadb-10a4-465f-bcf8-b0176c3606ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:10.231406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:10.231406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:10.231406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:10.231406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:10.231406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:10.231406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:10.231406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:10.231406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:10.231406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:10.231406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:10.231406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:10.231406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:10.231406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:10.231406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:10.231406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:10.235227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:10.235227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:10.235227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:10.235227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:10.235227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:10.235227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:10.235227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:10.235227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:10.235227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:10.235227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:10.235227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:10.235227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:10.235227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:10.235227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:10.235227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:10.235227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:10.236107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:10.236107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:10.236107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:10.236107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:10.236107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:10.236107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:10.236107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:10.236107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:10.236107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:10.236107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:10.236107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:10.236107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:10.236107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 330/647] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:59:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:10.236107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ccadb-10a4-465f-bcf8-b0176c3606ca None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:10.288470 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:10.304778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 318/648] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 03:59:10 2022] PUT /identity/v3/projects/68d376986e854a4bb334676572340b4c/users/6d8e1f1e3c7e42cc9fa6f82aaac55129/roles/3073b5cac26f43089c9ed8c6a5102cff => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:59:10.305769 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb60f0-6b46-490c-8a95-557a7b582e74 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:10.364736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ccadb-10a4-465f-bcf8-b0176c3606ca None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:10.410857 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8233b-75ea-4315-b0b2-615d41541e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:10.411674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8233b-75ea-4315-b0b2-615d41541e40 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:10.411964 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8233b-75ea-4315-b0b2-615d41541e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:10.412246 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8233b-75ea-4315-b0b2-615d41541e40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:10.425198 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8233b-75ea-4315-b0b2-615d41541e40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:10.426641 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8233b-75ea-4315-b0b2-615d41541e40 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:10.446194 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb60f0-6b46-490c-8a95-557a7b582e74 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:10.446569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-afe8233b-75ea-4315-b0b2-615d41541e40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:10.446569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:10.446569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:10.446569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:10.446569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:10.446569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:10.446569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:10.446569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:10.446569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:10.446569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:10.446569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:10.446569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:10.446569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:10.446569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:10.446569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:10.447459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:10.447459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:10.447459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:10.447459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:10.447459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:10.447459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:10.447459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:10.447459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:10.447459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:10.447459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:10.447459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:10.447459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:10.447459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:10.447459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:10.447459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:10.447459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:10.447459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:10.448382 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:10.448382 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:10.448382 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:10.448382 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:10.452876 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 331/649] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:10.453540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8233b-75ea-4315-b0b2-615d41541e40 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:10.474705 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:10.490697 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243794e-6dd0-4881-b8b7-5a9487a8e164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:10.491396 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243794e-6dd0-4881-b8b7-5a9487a8e164 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:10.491679 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243794e-6dd0-4881-b8b7-5a9487a8e164 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:10.491951 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243794e-6dd0-4881-b8b7-5a9487a8e164 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:10.492959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243794e-6dd0-4881-b8b7-5a9487a8e164 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:10.493367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243794e-6dd0-4881-b8b7-5a9487a8e164 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:10.497006 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243794e-6dd0-4881-b8b7-5a9487a8e164 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:10.520047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 319/650] 10.222.0.16 () {56 vars in 1137 bytes} [Fri Jun 24 03:59:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:10.521244 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243794e-6dd0-4881-b8b7-5a9487a8e164 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:10.568438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8233b-75ea-4315-b0b2-615d41541e40 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:10.571499 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:10.605390 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d7346-8acf-446a-a949-1014fff94f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:10.606068 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d7346-8acf-446a-a949-1014fff94f8e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:10.606360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d7346-8acf-446a-a949-1014fff94f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:10.606686 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d7346-8acf-446a-a949-1014fff94f8e None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:10.607441 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d7346-8acf-446a-a949-1014fff94f8e None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:10.607758 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d7346-8acf-446a-a949-1014fff94f8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:10.609147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d7346-8acf-446a-a949-1014fff94f8e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:10.618108 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 332/651] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 03:59:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:10.623102 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d7346-8acf-446a-a949-1014fff94f8e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:10.663424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243794e-6dd0-4881-b8b7-5a9487a8e164 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:10.712210 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c0c6a-23bb-4578-bdd4-9bffbbe288df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:10.713590 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c0c6a-23bb-4578-bdd4-9bffbbe288df None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:10.714530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c0c6a-23bb-4578-bdd4-9bffbbe288df None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:10.715025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c0c6a-23bb-4578-bdd4-9bffbbe288df None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:10.715956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c0c6a-23bb-4578-bdd4-9bffbbe288df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:10.716490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c0c6a-23bb-4578-bdd4-9bffbbe288df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:10.718961 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c0c6a-23bb-4578-bdd4-9bffbbe288df None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:10.729842 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d7346-8acf-446a-a949-1014fff94f8e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:10.736217 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 320/652] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:10.737983 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c0c6a-23bb-4578-bdd4-9bffbbe288df None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:10.777879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acd271-cde4-4df3-a431-f1539f86a915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:10.778591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acd271-cde4-4df3-a431-f1539f86a915 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:10.778884 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acd271-cde4-4df3-a431-f1539f86a915 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:10.779168 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11acd271-cde4-4df3-a431-f1539f86a915 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:10.787410 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:10.832719 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:10.868102 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c0c6a-23bb-4578-bdd4-9bffbbe288df None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:10.871630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11acd271-cde4-4df3-a431-f1539f86a915 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:10.873343 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11acd271-cde4-4df3-a431-f1539f86a915 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:10.910738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f837663-4bf2-496d-8b4a-e65a1ea9aa14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:10.911691 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f837663-4bf2-496d-8b4a-e65a1ea9aa14 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:10.911989 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f837663-4bf2-496d-8b4a-e65a1ea9aa14 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:10.912270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f837663-4bf2-496d-8b4a-e65a1ea9aa14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:10.925567 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f837663-4bf2-496d-8b4a-e65a1ea9aa14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:10.927726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f837663-4bf2-496d-8b4a-e65a1ea9aa14 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:10.950255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-8f837663-4bf2-496d-8b4a-e65a1ea9aa14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:10.950255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:10.950255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:10.950255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:10.950255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:10.950255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:10.950255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:10.950255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:10.950255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:10.950255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:10.950255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:10.950255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:10.950255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:10.950255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:10.950255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:10.951218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:10.951218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:10.951218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:10.951218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:10.951218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:10.951218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:10.951218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:10.951218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:10.951218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:10.951218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:10.951218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:10.951218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:10.951218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:10.951218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:10.951218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:10.951218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:10.951218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:10.952216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:10.952216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:10.952216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:10.952216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:10.952216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 321/653] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:10.952913 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f837663-4bf2-496d-8b4a-e65a1ea9aa14 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:10.997111 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 333/654] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:59:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:59:10.998104 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acd271-cde4-4df3-a431-f1539f86a915 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:11.066887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:11.079328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f837663-4bf2-496d-8b4a-e65a1ea9aa14 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:11.117017 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:11.121028 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2b22c-9eaa-45f9-ac34-13d55e64fe21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:11.122180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2b22c-9eaa-45f9-ac34-13d55e64fe21 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:11.122923 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2b22c-9eaa-45f9-ac34-13d55e64fe21 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:11.123616 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2b22c-9eaa-45f9-ac34-13d55e64fe21 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:11.129115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11acd271-cde4-4df3-a431-f1539f86a915 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:11.136637 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2b22c-9eaa-45f9-ac34-13d55e64fe21 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:11.138778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2b22c-9eaa-45f9-ac34-13d55e64fe21 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:11.159025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-0cd2b22c-9eaa-45f9-ac34-13d55e64fe21 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 03:59:11.159025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:11.159025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:11.159025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:11.159025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:11.159025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:11.159025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:11.159025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:11.159025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:11.159025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:11.159025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:11.159025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:11.159025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:11.159025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:11.159025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:11.159025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:11.160172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:11.160172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:11.160172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:11.160172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:11.160172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:11.160172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:11.160172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:11.160172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:11.160172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:11.160172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:11.160172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:11.160172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:11.160172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:11.160172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:11.160172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:11.160172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:11.160172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:11.160172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:11.161128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:11.161128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:11.161128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:11.161128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:11.161128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:11.161128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:11.161128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:11.161128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:11.161128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 03:59:11.161128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:11.168724 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 322/655] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:59:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:11.168724 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2b22c-9eaa-45f9-ac34-13d55e64fe21 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:11.173390 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35c90f-8889-4fdf-95a7-31f5190b885b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:11.174613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35c90f-8889-4fdf-95a7-31f5190b885b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:11.175168 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35c90f-8889-4fdf-95a7-31f5190b885b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:11.175660 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35c90f-8889-4fdf-95a7-31f5190b885b None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:11.177140 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35c90f-8889-4fdf-95a7-31f5190b885b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:11.177683 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35c90f-8889-4fdf-95a7-31f5190b885b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:11.180170 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35c90f-8889-4fdf-95a7-31f5190b885b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:11.200885 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 334/656] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:11.200885 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35c90f-8889-4fdf-95a7-31f5190b885b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:11.278224 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2b22c-9eaa-45f9-ac34-13d55e64fe21 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:11.307383 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35c90f-8889-4fdf-95a7-31f5190b885b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:11.315242 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c6f31-da9b-451c-a2b3-de12fd268648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:11.316290 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c6f31-da9b-451c-a2b3-de12fd268648 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:11.316780 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c6f31-da9b-451c-a2b3-de12fd268648 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:11.317260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c6f31-da9b-451c-a2b3-de12fd268648 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:11.328614 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c6f31-da9b-451c-a2b3-de12fd268648 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:11.330401 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64c6f31-da9b-451c-a2b3-de12fd268648 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:11.334856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:11.339379 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:11.344353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a5359-a538-4136-a38d-db7ef96f0ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:11.345037 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a5359-a538-4136-a38d-db7ef96f0ebf None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:11.350231 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a5359-a538-4136-a38d-db7ef96f0ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:11.350231 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a5359-a538-4136-a38d-db7ef96f0ebf None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:11.350231 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a5359-a538-4136-a38d-db7ef96f0ebf None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:11.350231 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a5359-a538-4136-a38d-db7ef96f0ebf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:11.350231 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a5359-a538-4136-a38d-db7ef96f0ebf None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:11.350602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-c64c6f31-da9b-451c-a2b3-de12fd268648 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:11.350602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:11.350602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:11.350602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:11.350602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:11.350602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:11.350602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:11.350602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:11.350602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:11.350602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:11.350602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:11.350602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:11.350602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:11.350602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:11.350602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:11.351424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:11.351424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:11.351424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:11.351424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:11.351424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:11.351424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:11.351424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:11.351424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:11.351424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:11.351424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:11.351424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:11.351424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:11.351424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:11.351424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:11.351424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:11.351424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:11.351424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:11.352341 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:11.352341 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:11.352341 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:11.352341 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:11.354450 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 323/657] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:11.355523 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c6f31-da9b-451c-a2b3-de12fd268648 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:11.373182 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 335/658] 10.222.0.16 () {56 vars in 1137 bytes} [Fri Jun 24 03:59:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:11.373182 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a5359-a538-4136-a38d-db7ef96f0ebf None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:11.465670 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64c6f31-da9b-451c-a2b3-de12fd268648 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:11.483334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a5359-a538-4136-a38d-db7ef96f0ebf None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:11.503705 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967468d-4dfa-4c22-a034-88db285b5e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:11.505277 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967468d-4dfa-4c22-a034-88db285b5e30 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:11.505768 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967468d-4dfa-4c22-a034-88db285b5e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:11.506245 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967468d-4dfa-4c22-a034-88db285b5e30 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:11.515643 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967468d-4dfa-4c22-a034-88db285b5e30 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:11.518434 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967468d-4dfa-4c22-a034-88db285b5e30 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:11.523346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b023b16-f6b8-43a2-971a-6cf23cd76065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:11.524029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b023b16-f6b8-43a2-971a-6cf23cd76065 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:11.524320 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b023b16-f6b8-43a2-971a-6cf23cd76065 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:11.524605 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b023b16-f6b8-43a2-971a-6cf23cd76065 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:11.525347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b023b16-f6b8-43a2-971a-6cf23cd76065 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:11.525664 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b023b16-f6b8-43a2-971a-6cf23cd76065 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:11.527067 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b023b16-f6b8-43a2-971a-6cf23cd76065 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:11.536233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-0967468d-4dfa-4c22-a034-88db285b5e30 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 03:59:11.536233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:11.536233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:11.536233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:11.536233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:11.536233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:11.536233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:11.536233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:11.536233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:11.536233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:11.536233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:11.536233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:11.536233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:11.536233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:11.536233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:11.536233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:11.537180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:11.537180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:11.537180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:11.537180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:11.537180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:11.537180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:11.537180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:11.537180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:11.537180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:11.537180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:11.537180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:11.537180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:11.537180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:11.537180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:11.537180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:11.537180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:11.537180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:11.537180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:11.538127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:11.538127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:11.538127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:11.538127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:11.538127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:11.538127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:11.538127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:11.538127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:11.538127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 03:59:11.538127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:11.539385 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 336/659] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:11.540289 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b023b16-f6b8-43a2-971a-6cf23cd76065 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:11.545675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 324/660] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:59:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:11.546844 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967468d-4dfa-4c22-a034-88db285b5e30 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:11.590882 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:11.623389 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:11.654481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b023b16-f6b8-43a2-971a-6cf23cd76065 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:11.665935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967468d-4dfa-4c22-a034-88db285b5e30 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:11.693207 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689d9bb-17d1-4cde-97bc-8c15a2a98265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:11.694174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689d9bb-17d1-4cde-97bc-8c15a2a98265 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:11.694764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689d9bb-17d1-4cde-97bc-8c15a2a98265 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:11.695262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689d9bb-17d1-4cde-97bc-8c15a2a98265 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:11.700115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37883d-c389-4bb3-99e4-26582b9f7476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:11.700900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37883d-c389-4bb3-99e4-26582b9f7476 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:11.701185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37883d-c389-4bb3-99e4-26582b9f7476 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:11.701463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37883d-c389-4bb3-99e4-26582b9f7476 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:11.709167 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37883d-c389-4bb3-99e4-26582b9f7476 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:11.710723 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37883d-c389-4bb3-99e4-26582b9f7476 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:11.725047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-ed37883d-c389-4bb3-99e4-26582b9f7476 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:11.725047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:11.725047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:11.725047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:11.725047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:11.725047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:11.725047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:11.725047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:11.725047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:11.725047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:11.725047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:11.725047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:11.725047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:11.725047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:11.725047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:11.725971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:11.725971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:11.725971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:11.725971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:11.725971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:11.725971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:11.725971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:11.725971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:11.725971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:11.725971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:11.725971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:11.725971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:11.725971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:11.725971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:11.725971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:11.725971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:11.725971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:11.726982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:11.726982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:11.726982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:11.726982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:11.726982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:11.726982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:11.726982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:11.726982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:11.726982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 325/661] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:11.727483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37883d-c389-4bb3-99e4-26582b9f7476 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:11.773172 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689d9bb-17d1-4cde-97bc-8c15a2a98265 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:11.775084 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e689d9bb-17d1-4cde-97bc-8c15a2a98265 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:11.823157 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:11.829990 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37883d-c389-4bb3-99e4-26582b9f7476 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:11.851335 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:11.866767 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c1568-373d-4517-95f1-4632ec33277e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:11.867439 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c1568-373d-4517-95f1-4632ec33277e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:11.868151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c1568-373d-4517-95f1-4632ec33277e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:11.868443 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c1568-373d-4517-95f1-4632ec33277e None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:11.869830 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c1568-373d-4517-95f1-4632ec33277e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:11.870146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c1568-373d-4517-95f1-4632ec33277e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:11.872382 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7c1568-373d-4517-95f1-4632ec33277e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:11.881799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 337/662] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:59:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:59:11.883283 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689d9bb-17d1-4cde-97bc-8c15a2a98265 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:11.892866 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 326/663] 10.222.0.16 () {56 vars in 1139 bytes} [Fri Jun 24 03:59:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:11.894304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691bee4-e20e-470f-945c-2b98fd8f906d None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:11.894630 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691bee4-e20e-470f-945c-2b98fd8f906d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:11.894921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691bee4-e20e-470f-945c-2b98fd8f906d None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:11.896450 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 327/664] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:11.896850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691bee4-e20e-470f-945c-2b98fd8f906d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:11.997923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689d9bb-17d1-4cde-97bc-8c15a2a98265 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:12.010755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691bee4-e20e-470f-945c-2b98fd8f906d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:12.040608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27831fd-fab9-4223-b9a2-00cb9b473f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:12.041363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27831fd-fab9-4223-b9a2-00cb9b473f70 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:12.041724 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27831fd-fab9-4223-b9a2-00cb9b473f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:12.042075 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27831fd-fab9-4223-b9a2-00cb9b473f70 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:12.043110 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27831fd-fab9-4223-b9a2-00cb9b473f70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:12.043492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27831fd-fab9-4223-b9a2-00cb9b473f70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:12.045073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27831fd-fab9-4223-b9a2-00cb9b473f70 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:12.051547 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeadfc6-8ff5-40fd-afe3-ca46349752a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:12.052551 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeadfc6-8ff5-40fd-afe3-ca46349752a8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:12.052745 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeadfc6-8ff5-40fd-afe3-ca46349752a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:12.053028 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeadfc6-8ff5-40fd-afe3-ca46349752a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:12.057663 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 338/665] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:12.063099 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27831fd-fab9-4223-b9a2-00cb9b473f70 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:12.064119 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeadfc6-8ff5-40fd-afe3-ca46349752a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:12.065705 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeadfc6-8ff5-40fd-afe3-ca46349752a8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:12.067520 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:12.085347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-7eeadfc6-8ff5-40fd-afe3-ca46349752a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:12.085347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:12.085347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:12.085347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:12.085347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:12.085347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:12.085347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:12.085347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:12.085347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:12.085347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:12.085347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:12.085347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:12.085347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:12.085347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:12.085347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:12.087293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:12.087293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:12.087293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:12.087293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:12.087293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:12.087293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:12.087293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:12.087293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:12.087293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:12.087293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:12.087293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:12.087293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:12.087293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:12.087293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:12.087293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:12.087293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:12.087293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:12.088233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:12.088233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:12.088233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:12.088233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:12.091313 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 328/666] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:12.092409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16d8c3-f310-4c0a-8f24-fc45eb48f2de None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:12.092709 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16d8c3-f310-4c0a-8f24-fc45eb48f2de None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:12.092992 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16d8c3-f310-4c0a-8f24-fc45eb48f2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:12.109809 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ed16d8c3-f310-4c0a-8f24-fc45eb48f2de None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:12.111028 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ed16d8c3-f310-4c0a-8f24-fc45eb48f2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:12.135945 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:12.173031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27831fd-fab9-4223-b9a2-00cb9b473f70 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:12.185760 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 329/667] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:59:12.187510 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16d8c3-f310-4c0a-8f24-fc45eb48f2de None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:12.208763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7236107-59ee-4112-80a3-13e03ae7506e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:12.209618 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7236107-59ee-4112-80a3-13e03ae7506e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:12.209915 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7236107-59ee-4112-80a3-13e03ae7506e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:12.210197 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7236107-59ee-4112-80a3-13e03ae7506e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:12.286088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7236107-59ee-4112-80a3-13e03ae7506e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:12.287937 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7236107-59ee-4112-80a3-13e03ae7506e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:12.291829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:12.311415 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16d8c3-f310-4c0a-8f24-fc45eb48f2de None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:12.350052 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e9065-f393-442e-b8ba-f67178a236e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:12.350839 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e9065-f393-442e-b8ba-f67178a236e2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:12.351021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e9065-f393-442e-b8ba-f67178a236e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:12.351307 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e9065-f393-442e-b8ba-f67178a236e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:12.352141 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e9065-f393-442e-b8ba-f67178a236e2 None admin] RBAC: Adding query filter params (role.id=3073b5cac26f43089c9ed8c6a5102cff, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=0fbec7435e0841a5b96946dca814e0c6) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:12.352463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e9065-f393-442e-b8ba-f67178a236e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:12.353919 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23e9065-f393-442e-b8ba-f67178a236e2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:12.370423 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 330/668] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:59:12 2022] GET /identity/v3/role_assignments?user.id=0fbec7435e0841a5b96946dca814e0c6&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=3073b5cac26f43089c9ed8c6a5102cff => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:12.370423 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e9065-f393-442e-b8ba-f67178a236e2 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:12.398796 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:12.413912 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 339/669] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:59:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:59:12.415052 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7236107-59ee-4112-80a3-13e03ae7506e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:12.492977 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e9065-f393-442e-b8ba-f67178a236e2 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:12.532601 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7236107-59ee-4112-80a3-13e03ae7506e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:12.535036 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868efac2-9127-4a82-88ee-f639044962fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:12.535781 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868efac2-9127-4a82-88ee-f639044962fb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:12.536066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868efac2-9127-4a82-88ee-f639044962fb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:12.536340 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868efac2-9127-4a82-88ee-f639044962fb None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:12.537333 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868efac2-9127-4a82-88ee-f639044962fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:12.537456 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868efac2-9127-4a82-88ee-f639044962fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:12.537879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:12.538794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868efac2-9127-4a82-88ee-f639044962fb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:12.552128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 331/670] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:12.553261 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868efac2-9127-4a82-88ee-f639044962fb None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:12.576175 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f307cbd0-1c1f-41fa-976d-8819f86282e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:12.576649 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307cbd0-1c1f-41fa-976d-8819f86282e3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:12.577048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307cbd0-1c1f-41fa-976d-8819f86282e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:12.577237 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f307cbd0-1c1f-41fa-976d-8819f86282e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:12.590402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f307cbd0-1c1f-41fa-976d-8819f86282e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:12.597418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-f307cbd0-1c1f-41fa-976d-8819f86282e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:59:12.597418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:12.597418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:12.597418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:12.597418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:12.597418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:12.597418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:12.597418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:12.597418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:12.597418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:12.597418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:12.597418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:12.597418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:12.597418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:12.597418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:12.597418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:12.598421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:12.598421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:12.598421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:12.598421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:12.598421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:12.598421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:12.598421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:12.598421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:12.598421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:12.598421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:12.598421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:12.598421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:12.598421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:12.598421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:12.598421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:12.598421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:12.599293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:12.599293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:12.599293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:12.599293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:12.599293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:12.599293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:12.599293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:12.599293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:12.599293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:12.599293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:12.599293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:59:12.599293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:12.599293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 340/671] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:59:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:12.600268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f555d26-0929-468c-918c-6e29a023c6f6 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:12.600491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f555d26-0929-468c-918c-6e29a023c6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:12.600769 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f555d26-0929-468c-918c-6e29a023c6f6 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:12.602276 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 341/672] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:12.607019 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f555d26-0929-468c-918c-6e29a023c6f6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:12.660017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:12.674926 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868efac2-9127-4a82-88ee-f639044962fb None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:12.709790 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde20b7b-bdba-46f9-8a79-15c205d102c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:12.710731 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde20b7b-bdba-46f9-8a79-15c205d102c5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:12.711133 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde20b7b-bdba-46f9-8a79-15c205d102c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:12.711412 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde20b7b-bdba-46f9-8a79-15c205d102c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:12.717608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f555d26-0929-468c-918c-6e29a023c6f6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:12.723336 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde20b7b-bdba-46f9-8a79-15c205d102c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:12.724918 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde20b7b-bdba-46f9-8a79-15c205d102c5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:12.744865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-bde20b7b-bdba-46f9-8a79-15c205d102c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:12.744865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:12.744865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:12.744865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:12.744865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:12.744865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:12.744865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:12.744865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:12.744865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:12.744865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:12.744865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:12.744865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:12.744865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:12.744865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:12.744865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:12.745829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:12.745829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:12.745829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:12.745829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:12.745829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:12.745829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:12.745829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:12.745829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:12.745829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:12.745829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:12.745829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:12.745829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:12.745829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:12.745829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:12.745829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:12.745829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:12.745829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:12.747230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:12.747230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:12.747230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:12.747230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:12.747230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 332/673] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:12.747230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde20b7b-bdba-46f9-8a79-15c205d102c5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:12.757782 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd523c3-d81f-4543-a466-a8dc9a5ea74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:12.758606 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd523c3-d81f-4543-a466-a8dc9a5ea74e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:12.758896 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd523c3-d81f-4543-a466-a8dc9a5ea74e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:12.759175 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd523c3-d81f-4543-a466-a8dc9a5ea74e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:12.768022 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd523c3-d81f-4543-a466-a8dc9a5ea74e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:12.769907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd523c3-d81f-4543-a466-a8dc9a5ea74e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:12.786206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-efd523c3-d81f-4543-a466-a8dc9a5ea74e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:12.786206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:12.786206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:12.786206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:12.786206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:12.786206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:12.786206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:12.786206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:12.786206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:12.786206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:12.786206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:12.786206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:12.786206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:12.786206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:12.786206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:12.787315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:12.787315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:12.787315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:12.787315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:12.787315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:12.787315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:12.787315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:12.787315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:12.787315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:12.787315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:12.787315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:12.787315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:12.787315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:12.787315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:12.787315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:12.787315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:12.787315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:12.788221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:12.788221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:12.788221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:12.788221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:12.788221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:12.788221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:12.788221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:12.788221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:12.788221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 342/674] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:12.789276 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f57a1-293c-4fcf-a581-4e4e02230508 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:12.789276 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f57a1-293c-4fcf-a581-4e4e02230508 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:12.789538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f57a1-293c-4fcf-a581-4e4e02230508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:12.806445 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-427f57a1-293c-4fcf-a581-4e4e02230508 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:12.807643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-427f57a1-293c-4fcf-a581-4e4e02230508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:12.830143 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:12.869209 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde20b7b-bdba-46f9-8a79-15c205d102c5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:12.884048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 343/675] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:59:12.884785 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f57a1-293c-4fcf-a581-4e4e02230508 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:12.919641 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa8cc7-a0b5-4c64-a7cb-5e1bec5d8611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:12.920336 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa8cc7-a0b5-4c64-a7cb-5e1bec5d8611 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:12.920626 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa8cc7-a0b5-4c64-a7cb-5e1bec5d8611 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:12.920925 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa8cc7-a0b5-4c64-a7cb-5e1bec5d8611 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:12.921657 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa8cc7-a0b5-4c64-a7cb-5e1bec5d8611 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:12.921973 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa8cc7-a0b5-4c64-a7cb-5e1bec5d8611 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:12.923384 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa8cc7-a0b5-4c64-a7cb-5e1bec5d8611 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:12.926345 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:12.933853 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 333/676] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:59:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:12.934776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa8cc7-a0b5-4c64-a7cb-5e1bec5d8611 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:12.997258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f57a1-293c-4fcf-a581-4e4e02230508 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:13.032857 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45e3e4-6170-4b10-a439-d9c6d1125480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:13.033682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45e3e4-6170-4b10-a439-d9c6d1125480 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:13.034049 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45e3e4-6170-4b10-a439-d9c6d1125480 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:13.034258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45e3e4-6170-4b10-a439-d9c6d1125480 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:13.035021 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45e3e4-6170-4b10-a439-d9c6d1125480 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:13.035334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45e3e4-6170-4b10-a439-d9c6d1125480 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:13.036720 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45e3e4-6170-4b10-a439-d9c6d1125480 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:13.049014 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 344/677] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:13.050087 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45e3e4-6170-4b10-a439-d9c6d1125480 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:13.058099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa8cc7-a0b5-4c64-a7cb-5e1bec5d8611 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:13.067713 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:13.097255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cccb0f-0622-4bb1-86c8-f50b5478ec1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:13.097922 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cccb0f-0622-4bb1-86c8-f50b5478ec1b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:13.098207 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cccb0f-0622-4bb1-86c8-f50b5478ec1b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:13.098537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cccb0f-0622-4bb1-86c8-f50b5478ec1b None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:13.099382 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cccb0f-0622-4bb1-86c8-f50b5478ec1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:13.099700 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cccb0f-0622-4bb1-86c8-f50b5478ec1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:13.101189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cccb0f-0622-4bb1-86c8-f50b5478ec1b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:13.113879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 334/678] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:13.119526 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cccb0f-0622-4bb1-86c8-f50b5478ec1b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:13.156660 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45e3e4-6170-4b10-a439-d9c6d1125480 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:13.183386 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:13.193146 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210862b-51ba-4812-ae81-b34171683096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:13.198474 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210862b-51ba-4812-ae81-b34171683096 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:13.198474 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210862b-51ba-4812-ae81-b34171683096 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:13.198474 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c210862b-51ba-4812-ae81-b34171683096 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:13.205094 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c210862b-51ba-4812-ae81-b34171683096 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:13.213803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-c210862b-51ba-4812-ae81-b34171683096 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:13.213803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:13.213803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:13.213803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:13.213803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:13.213803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:13.213803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:13.213803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:13.213803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:13.213803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:13.213803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:13.213803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:13.213803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:13.213803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:13.213803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:13.213803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:13.214799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:13.214799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:13.214799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:13.214799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:13.214799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:13.214799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:13.214799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:13.214799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:13.214799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:13.214799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:13.214799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:13.214799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:13.214799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:13.214799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:13.214799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:13.214799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:13.215661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:13.215661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:13.215661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:13.215661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:13.215661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:13.215661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:13.215661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:13.215661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:13.215661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:13.215661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:13.215661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:13.215661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:13.215661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 345/679] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:59:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:13.216368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210862b-51ba-4812-ae81-b34171683096 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:13.234358 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cccb0f-0622-4bb1-86c8-f50b5478ec1b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:13.270670 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ba237-ae2f-4fe6-9343-d138f5c491be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:13.271612 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ba237-ae2f-4fe6-9343-d138f5c491be None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:13.271909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ba237-ae2f-4fe6-9343-d138f5c491be None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:13.272189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ba237-ae2f-4fe6-9343-d138f5c491be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:13.283859 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ba237-ae2f-4fe6-9343-d138f5c491be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:13.285319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ba237-ae2f-4fe6-9343-d138f5c491be None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:13.299996 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:13.304160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-6f8ba237-ae2f-4fe6-9343-d138f5c491be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:13.304160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:13.304160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:13.304160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:13.304160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:13.304160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:13.304160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:13.304160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:13.304160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:13.304160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:13.304160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:13.304160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:13.304160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:13.304160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:13.304160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:13.305085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:13.305085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:13.305085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:13.305085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:13.305085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:13.305085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:13.305085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:13.305085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:13.305085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:13.305085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:13.305085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:13.305085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:13.305085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:13.305085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:13.305085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:13.305085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:13.305085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:13.305974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:13.305974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:13.305974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:13.305974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:13.305974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 335/680] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:13.306259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ba237-ae2f-4fe6-9343-d138f5c491be None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:13.331486 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c210862b-51ba-4812-ae81-b34171683096 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:13.367423 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b495ec-a984-4f5c-8cf4-364171280b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:13.368221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b495ec-a984-4f5c-8cf4-364171280b08 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:13.368508 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b495ec-a984-4f5c-8cf4-364171280b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:13.368794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b495ec-a984-4f5c-8cf4-364171280b08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:13.375714 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b495ec-a984-4f5c-8cf4-364171280b08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:13.378125 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b495ec-a984-4f5c-8cf4-364171280b08 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:13.392158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-b2b495ec-a984-4f5c-8cf4-364171280b08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:13.392158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:13.392158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:13.392158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:13.392158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:13.392158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:13.392158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:13.392158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:13.392158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:13.392158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:13.392158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:13.392158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:13.392158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:13.392158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:13.392158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:13.393488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:13.393488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:13.393488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:13.393488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:13.393488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:13.393488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:13.393488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:13.393488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:13.393488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:13.393488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:13.393488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:13.393488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:13.393488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:13.393488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:13.393488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:13.393488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:13.393488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:13.394416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:13.394416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:13.394416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:13.394416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:13.394416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:13.394416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:13.394416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:13.394416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:13.394416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 346/681] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:13.394416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b495ec-a984-4f5c-8cf4-364171280b08 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:13.419178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ba237-ae2f-4fe6-9343-d138f5c491be None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:13.430955 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:13.456785 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee5c08-0a57-49c4-84d3-eb409c3ada27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:13.457380 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee5c08-0a57-49c4-84d3-eb409c3ada27 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:13.457759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee5c08-0a57-49c4-84d3-eb409c3ada27 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:13.457951 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee5c08-0a57-49c4-84d3-eb409c3ada27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:13.458799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee5c08-0a57-49c4-84d3-eb409c3ada27 None admin] RBAC: Adding query filter params (role.id=3073b5cac26f43089c9ed8c6a5102cff, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=61459e15d327405b88a2fdfc1fc0fecd) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:13.459264 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee5c08-0a57-49c4-84d3-eb409c3ada27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:13.460549 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee5c08-0a57-49c4-84d3-eb409c3ada27 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:13.474417 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 336/682] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:59:13 2022] GET /identity/v3/role_assignments?user.id=61459e15d327405b88a2fdfc1fc0fecd&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=3073b5cac26f43089c9ed8c6a5102cff => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:13.474417 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee5c08-0a57-49c4-84d3-eb409c3ada27 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:13.523993 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b495ec-a984-4f5c-8cf4-364171280b08 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:13.562802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b71bc-0e3a-4995-82eb-63654187f83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:13.563506 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b71bc-0e3a-4995-82eb-63654187f83a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:13.563797 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b71bc-0e3a-4995-82eb-63654187f83a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:13.564084 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b71bc-0e3a-4995-82eb-63654187f83a None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:13.565690 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b71bc-0e3a-4995-82eb-63654187f83a None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:13.566130 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b71bc-0e3a-4995-82eb-63654187f83a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:13.567733 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b71bc-0e3a-4995-82eb-63654187f83a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:13.569682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:13.582581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 347/683] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 03:59:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:13.582581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b71bc-0e3a-4995-82eb-63654187f83a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:13.618405 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee5c08-0a57-49c4-84d3-eb409c3ada27 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:13.668758 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758c5c0-3048-4b3d-b1c7-241ceab513c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:13.669555 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758c5c0-3048-4b3d-b1c7-241ceab513c0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:13.669909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758c5c0-3048-4b3d-b1c7-241ceab513c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:13.670258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758c5c0-3048-4b3d-b1c7-241ceab513c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:13.671098 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758c5c0-3048-4b3d-b1c7-241ceab513c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:13.671474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758c5c0-3048-4b3d-b1c7-241ceab513c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:13.672919 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c758c5c0-3048-4b3d-b1c7-241ceab513c0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:13.686008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 337/684] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:13.686748 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758c5c0-3048-4b3d-b1c7-241ceab513c0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:13.713164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b71bc-0e3a-4995-82eb-63654187f83a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:13.728845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:13.757891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5d77a-e098-4019-a10a-040ad07791b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:13.758771 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5d77a-e098-4019-a10a-040ad07791b3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:13.759069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5d77a-e098-4019-a10a-040ad07791b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:13.759353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5d77a-e098-4019-a10a-040ad07791b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:13.760155 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5d77a-e098-4019-a10a-040ad07791b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:13.760558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5d77a-e098-4019-a10a-040ad07791b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:13.761897 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5d77a-e098-4019-a10a-040ad07791b3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:13.776998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 348/685] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:13.777759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5d77a-e098-4019-a10a-040ad07791b3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:13.817321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c758c5c0-3048-4b3d-b1c7-241ceab513c0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:13.848285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:13.864509 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e27fe6-6adb-4358-9d44-6b39125191f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:13.866164 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e27fe6-6adb-4358-9d44-6b39125191f5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:13.866479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e27fe6-6adb-4358-9d44-6b39125191f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:13.866762 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e27fe6-6adb-4358-9d44-6b39125191f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:13.880004 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e27fe6-6adb-4358-9d44-6b39125191f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:13.881464 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e27fe6-6adb-4358-9d44-6b39125191f5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:13.903035 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5d77a-e098-4019-a10a-040ad07791b3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:13.907525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-83e27fe6-6adb-4358-9d44-6b39125191f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:13.907525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:13.907525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:13.907525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:13.907525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:13.907525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:13.907525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:13.907525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:13.907525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:13.907525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:13.907525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:13.907525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:13.907525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:13.907525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:13.907525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:13.908967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:13.908967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:13.908967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:13.908967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:13.908967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:13.908967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:13.908967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:13.908967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:13.908967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:13.908967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:13.908967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:13.908967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:13.908967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:13.908967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:13.908967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:13.908967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:13.908967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:13.910082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:13.910082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:13.910082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:13.910082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:13.917766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 338/686] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:13.919238 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e27fe6-6adb-4358-9d44-6b39125191f5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:13.944725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978d06b-4017-4874-bb1a-4400f2c48310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:13.945530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978d06b-4017-4874-bb1a-4400f2c48310 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:13.945812 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978d06b-4017-4874-bb1a-4400f2c48310 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:13.946086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4978d06b-4017-4874-bb1a-4400f2c48310 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:13.956545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978d06b-4017-4874-bb1a-4400f2c48310 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:13.958266 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4978d06b-4017-4874-bb1a-4400f2c48310 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:13.981047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-4978d06b-4017-4874-bb1a-4400f2c48310 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 03:59:13.981047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:13.981047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:13.981047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:13.981047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:13.981047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:13.981047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:13.981047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:13.981047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:13.981047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:13.981047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:13.981047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:13.981047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:13.981047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:13.981047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:13.981047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:13.982074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:13.982074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:13.982074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:13.982074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:13.982074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:13.982074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:13.982074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:13.982074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:13.982074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:13.982074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:13.982074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:13.982074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:13.982074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:13.982074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:13.982074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:13.982074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:13.982074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:13.982074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:13.983064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:13.983064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:13.983064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:13.983064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:13.983064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:13.983064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:13.983064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:13.983064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:13.983064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 03:59:13.983064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:13.983064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 349/687] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Jun 24 03:59:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:13.983064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978d06b-4017-4874-bb1a-4400f2c48310 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:14.037157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:14.055214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e27fe6-6adb-4358-9d44-6b39125191f5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:14.093560 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52869579-5a00-4f24-8e34-ab1a524c8b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:14.094225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52869579-5a00-4f24-8e34-ab1a524c8b2b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:14.094529 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52869579-5a00-4f24-8e34-ab1a524c8b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:14.094806 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52869579-5a00-4f24-8e34-ab1a524c8b2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:14.095635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52869579-5a00-4f24-8e34-ab1a524c8b2b None admin] RBAC: Adding query filter params (role.id=3073b5cac26f43089c9ed8c6a5102cff, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=fd98b54f587f4b42b4dd0a058b30c684) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:14.096507 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52869579-5a00-4f24-8e34-ab1a524c8b2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:14.098307 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52869579-5a00-4f24-8e34-ab1a524c8b2b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:14.110398 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 339/688] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:59:13 2022] GET /identity/v3/role_assignments?user.id=fd98b54f587f4b42b4dd0a058b30c684&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=3073b5cac26f43089c9ed8c6a5102cff => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:14.110398 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52869579-5a00-4f24-8e34-ab1a524c8b2b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:14.126404 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4978d06b-4017-4874-bb1a-4400f2c48310 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:14.156250 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:14.199763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15abbe1-aabc-4137-85c8-a376db35d701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:14.200463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15abbe1-aabc-4137-85c8-a376db35d701 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:14.200753 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15abbe1-aabc-4137-85c8-a376db35d701 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:14.201036 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15abbe1-aabc-4137-85c8-a376db35d701 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:14.201787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15abbe1-aabc-4137-85c8-a376db35d701 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:14.202104 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15abbe1-aabc-4137-85c8-a376db35d701 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:14.203549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15abbe1-aabc-4137-85c8-a376db35d701 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:14.226213 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 350/689] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:14.229420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52869579-5a00-4f24-8e34-ab1a524c8b2b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:14.232041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15abbe1-aabc-4137-85c8-a376db35d701 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:14.301601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d991911-4042-4336-abe2-6a808f6279e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:14.302527 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d991911-4042-4336-abe2-6a808f6279e6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:14.302821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d991911-4042-4336-abe2-6a808f6279e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:14.303109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d991911-4042-4336-abe2-6a808f6279e6 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:14.304941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d991911-4042-4336-abe2-6a808f6279e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:14.305278 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d991911-4042-4336-abe2-6a808f6279e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:14.306787 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d991911-4042-4336-abe2-6a808f6279e6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:14.345286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 340/690] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Jun 24 03:59:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:14.353417 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d991911-4042-4336-abe2-6a808f6279e6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:14.366745 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:14.407335 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15abbe1-aabc-4137-85c8-a376db35d701 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:14.450907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd525935-e293-4cab-ad14-ddbb45b4a9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:14.452548 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd525935-e293-4cab-ad14-ddbb45b4a9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:14.453364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd525935-e293-4cab-ad14-ddbb45b4a9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:14.454165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd525935-e293-4cab-ad14-ddbb45b4a9a9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:14.464251 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd525935-e293-4cab-ad14-ddbb45b4a9a9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:14.466306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd525935-e293-4cab-ad14-ddbb45b4a9a9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:14.483759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-dd525935-e293-4cab-ad14-ddbb45b4a9a9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 03:59:14.483759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:14.483759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:14.483759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:14.483759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:14.483759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:14.483759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:14.483759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:14.483759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:14.483759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:14.483759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:14.483759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:14.483759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:14.483759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:14.483759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:14.483759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:14.484729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:14.484729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:14.484729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:14.484729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:14.484729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:14.484729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:14.484729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:14.484729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:14.484729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:14.484729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:14.484729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:14.484729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:14.484729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:14.484729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:14.484729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:14.484729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:14.484729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:14.484729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:14.485787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:14.485787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:14.485787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:14.485787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:14.485787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:14.485787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:14.485787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:14.485787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:14.485787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 03:59:14.485787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:14.485787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 351/691] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:59:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:14.493035 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd525935-e293-4cab-ad14-ddbb45b4a9a9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:14.498887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d991911-4042-4336-abe2-6a808f6279e6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:14.540844 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:14.552720 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10b115-9b4a-4862-87bb-9c5a638e514b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:14.553384 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10b115-9b4a-4862-87bb-9c5a638e514b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:14.553665 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10b115-9b4a-4862-87bb-9c5a638e514b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:14.553945 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10b115-9b4a-4862-87bb-9c5a638e514b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:14.634511 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd525935-e293-4cab-ad14-ddbb45b4a9a9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:14.639345 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10b115-9b4a-4862-87bb-9c5a638e514b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:14.640809 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10b115-9b4a-4862-87bb-9c5a638e514b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:14.651747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:14.681983 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c4ff9-c816-4eb3-b195-84103f02e54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:14.682777 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c4ff9-c816-4eb3-b195-84103f02e54a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:14.683067 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c4ff9-c816-4eb3-b195-84103f02e54a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:14.683701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c4ff9-c816-4eb3-b195-84103f02e54a None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:14.684556 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c4ff9-c816-4eb3-b195-84103f02e54a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:14.685184 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c4ff9-c816-4eb3-b195-84103f02e54a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:14.686966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c4ff9-c816-4eb3-b195-84103f02e54a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:14.714478 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 352/692] 10.222.0.16 () {56 vars in 1135 bytes} [Fri Jun 24 03:59:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:14.715860 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11437281-fd47-493f-8de8-569ca0dd263f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:14.716157 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11437281-fd47-493f-8de8-569ca0dd263f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:14.716440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11437281-fd47-493f-8de8-569ca0dd263f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:14.717980 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 353/693] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:14.720580 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11437281-fd47-493f-8de8-569ca0dd263f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:14.754908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 341/694] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:59:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:59:14.756151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc4acf-67a2-49cb-a926-910fd2a3bfb6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:14.756441 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc4acf-67a2-49cb-a926-910fd2a3bfb6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:14.756719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dc4acf-67a2-49cb-a926-910fd2a3bfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:14.778927 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-39dc4acf-67a2-49cb-a926-910fd2a3bfb6 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:14.780202 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-39dc4acf-67a2-49cb-a926-910fd2a3bfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:14.873473 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:14.874646 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 342/695] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:59:14.875591 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc4acf-67a2-49cb-a926-910fd2a3bfb6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:14.893950 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11437281-fd47-493f-8de8-569ca0dd263f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:14.939024 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:14.942114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68918612-fe17-466a-a872-b8bb57d1226d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:14.942802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68918612-fe17-466a-a872-b8bb57d1226d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:14.943489 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68918612-fe17-466a-a872-b8bb57d1226d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:14.943774 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68918612-fe17-466a-a872-b8bb57d1226d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:15.026040 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68918612-fe17-466a-a872-b8bb57d1226d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:15.026314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dc4acf-67a2-49cb-a926-910fd2a3bfb6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:15.027681 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68918612-fe17-466a-a872-b8bb57d1226d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:15.072965 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e19d47-046d-46a4-b3fc-f4c0b2bafe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:15.074139 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e19d47-046d-46a4-b3fc-f4c0b2bafe4e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:15.074650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e19d47-046d-46a4-b3fc-f4c0b2bafe4e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:15.075130 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e19d47-046d-46a4-b3fc-f4c0b2bafe4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:15.087497 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e19d47-046d-46a4-b3fc-f4c0b2bafe4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:15.089295 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e19d47-046d-46a4-b3fc-f4c0b2bafe4e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:15.108855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-f7e19d47-046d-46a4-b3fc-f4c0b2bafe4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:15.108855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:15.108855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:15.108855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:15.108855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:15.108855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:15.108855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:15.108855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:15.108855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:15.108855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:15.108855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:15.108855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:15.108855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:15.108855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:15.108855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:15.109761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:15.109761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:15.109761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:15.109761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:15.109761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:15.109761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:15.109761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:15.109761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:15.109761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:15.109761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:15.109761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:15.109761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:15.109761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:15.109761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:15.109761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:15.109761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:15.109761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:15.110684 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:15.110684 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:15.110684 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:15.110684 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:15.114031 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 343/696] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:15.115079 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e19d47-046d-46a4-b3fc-f4c0b2bafe4e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:15.154281 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:15.168893 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 354/697] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:59:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 03:59:15.168893 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68918612-fe17-466a-a872-b8bb57d1226d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:15.232782 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:15.248756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e19d47-046d-46a4-b3fc-f4c0b2bafe4e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:15.284639 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68918612-fe17-466a-a872-b8bb57d1226d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:15.295068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af1a6a-2a6d-4bc7-b014-69279e455971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:15.295884 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af1a6a-2a6d-4bc7-b014-69279e455971 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:15.296172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af1a6a-2a6d-4bc7-b014-69279e455971 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:15.296453 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af1a6a-2a6d-4bc7-b014-69279e455971 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:15.297569 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af1a6a-2a6d-4bc7-b014-69279e455971 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:15.300080 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af1a6a-2a6d-4bc7-b014-69279e455971 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:15.319232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-c3af1a6a-2a6d-4bc7-b014-69279e455971 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 03:59:15.319232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:15.319232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:15.319232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:15.319232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:15.319232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:15.319232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:15.319232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:15.319232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 03:59:15.319232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:15.319232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:15.319232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:15.319232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:15.319232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:15.319232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:15.319232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:15.320259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:15.320259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:15.320259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:15.320259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:15.320259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:15.320259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:15.320259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:15.320259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:15.320259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:15.320259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:15.320259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:15.320259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:15.320259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:15.320259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:15.320259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:15.320259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:15.321103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:15.321103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:15.321103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:15.321103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:15.321103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:15.321103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:15.321103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:15.321103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:15.321103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:15.321103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:15.321103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:15.321103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:15.321103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:15.321103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 03:59:15.321103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:15.325634 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 344/698] 10.222.0.16 () {56 vars in 1129 bytes} [Fri Jun 24 03:59:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:15.326920 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af1a6a-2a6d-4bc7-b014-69279e455971 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:15.331579 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddac5f-ae5f-41dd-ab40-471df0b5cb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:15.332535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddac5f-ae5f-41dd-ab40-471df0b5cb2b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:15.333032 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddac5f-ae5f-41dd-ab40-471df0b5cb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:15.333504 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ddac5f-ae5f-41dd-ab40-471df0b5cb2b None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:15.334474 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ddac5f-ae5f-41dd-ab40-471df0b5cb2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:15.335004 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ddac5f-ae5f-41dd-ab40-471df0b5cb2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:15.336576 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ddac5f-ae5f-41dd-ab40-471df0b5cb2b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:15.351844 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 355/699] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:15.353692 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddac5f-ae5f-41dd-ab40-471df0b5cb2b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:15.416932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:15.465196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af1a6a-2a6d-4bc7-b014-69279e455971 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:15.469412 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ddac5f-ae5f-41dd-ab40-471df0b5cb2b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:15.506342 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4e2c5-4cb3-406d-abfd-66caf49928a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:15.507582 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4e2c5-4cb3-406d-abfd-66caf49928a5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:15.508097 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4e2c5-4cb3-406d-abfd-66caf49928a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:15.508490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4e2c5-4cb3-406d-abfd-66caf49928a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:15.518978 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d42f3c-945c-49f1-857d-063e3f2eaadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:15.519941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d42f3c-945c-49f1-857d-063e3f2eaadc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:15.520409 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d42f3c-945c-49f1-857d-063e3f2eaadc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:15.520869 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d42f3c-945c-49f1-857d-063e3f2eaadc None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:15.522515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d42f3c-945c-49f1-857d-063e3f2eaadc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:15.523253 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d42f3c-945c-49f1-857d-063e3f2eaadc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:15.523877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4e2c5-4cb3-406d-abfd-66caf49928a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:15.524977 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d42f3c-945c-49f1-857d-063e3f2eaadc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:15.525540 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4e2c5-4cb3-406d-abfd-66caf49928a5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:15.536409 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 356/700] 10.222.0.16 () {56 vars in 1124 bytes} [Fri Jun 24 03:59:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:15.536409 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d42f3c-945c-49f1-857d-063e3f2eaadc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:15.549935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-5fb4e2c5-4cb3-406d-abfd-66caf49928a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:15.549935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:15.549935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:15.549935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:15.549935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:15.549935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:15.549935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:15.549935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:15.549935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:15.549935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:15.549935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:15.549935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:15.549935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:15.549935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:15.549935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:15.550895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:15.550895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:15.550895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:15.550895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:15.550895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:15.550895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:15.550895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:15.550895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:15.550895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:15.550895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:15.550895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:15.550895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:15.550895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:15.550895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:15.550895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:15.550895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:15.550895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:15.551827 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:15.551827 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:15.551827 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:15.551827 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:15.555689 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 345/701] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:15.556556 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4e2c5-4cb3-406d-abfd-66caf49928a5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:15.566329 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:15.667721 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d42f3c-945c-49f1-857d-063e3f2eaadc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:15.671687 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4e2c5-4cb3-406d-abfd-66caf49928a5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:15.690477 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:15.711532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102d3df-3959-4db6-95d2-69644778f19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:15.712540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102d3df-3959-4db6-95d2-69644778f19c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:15.712830 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102d3df-3959-4db6-95d2-69644778f19c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:15.713115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5102d3df-3959-4db6-95d2-69644778f19c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:15.713396 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ef97a-14f7-4076-a639-43e27174c5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:15.714053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ef97a-14f7-4076-a639-43e27174c5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:15.714408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ef97a-14f7-4076-a639-43e27174c5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:15.714745 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ef97a-14f7-4076-a639-43e27174c5b9 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:15.716000 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ef97a-14f7-4076-a639-43e27174c5b9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:15.716318 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ef97a-14f7-4076-a639-43e27174c5b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:15.717687 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33ef97a-14f7-4076-a639-43e27174c5b9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:15.720240 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102d3df-3959-4db6-95d2-69644778f19c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:15.722765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5102d3df-3959-4db6-95d2-69644778f19c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:15.725214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 346/702] 10.222.0.16 () {56 vars in 1124 bytes} [Fri Jun 24 03:59:15 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:15.726528 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ef97a-14f7-4076-a639-43e27174c5b9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:15.741929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-5102d3df-3959-4db6-95d2-69644778f19c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:15.741929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:15.741929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:15.741929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:15.741929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:15.741929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:15.741929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:15.741929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:15.741929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:15.741929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:15.741929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:15.741929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:15.741929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:15.741929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:15.741929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:15.742923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:15.742923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:15.742923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:15.742923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:15.742923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:15.742923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:15.742923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:15.742923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:15.742923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:15.742923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:15.742923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:15.742923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:15.742923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:15.742923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:15.742923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:15.742923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:15.742923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:15.743833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:15.743833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:15.743833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:15.743833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:15.743833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:15.743833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:15.743833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:15.743833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:15.743833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 357/703] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:15.749960 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102d3df-3959-4db6-95d2-69644778f19c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:15.850165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:15.886211 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33ef97a-14f7-4076-a639-43e27174c5b9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:15.919215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5102d3df-3959-4db6-95d2-69644778f19c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:15.924763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccbefa7-5d7a-4d50-9311-04e6bb916a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:15.925430 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbefa7-5d7a-4d50-9311-04e6bb916a9f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:15.925719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbefa7-5d7a-4d50-9311-04e6bb916a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:15.926028 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccbefa7-5d7a-4d50-9311-04e6bb916a9f None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:15.926789 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccbefa7-5d7a-4d50-9311-04e6bb916a9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:15.927102 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccbefa7-5d7a-4d50-9311-04e6bb916a9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:15.928501 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccbefa7-5d7a-4d50-9311-04e6bb916a9f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:15.942639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 347/704] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:15.943963 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac202e5-7502-4fc3-9436-1fffcb71e10c None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:15.944249 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac202e5-7502-4fc3-9436-1fffcb71e10c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:15.944528 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac202e5-7502-4fc3-9436-1fffcb71e10c None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:15.950161 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 348/705] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:15.950161 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac202e5-7502-4fc3-9436-1fffcb71e10c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:15.977898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396f0b2-17c4-4abf-a5c8-8032482475f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:15.979467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396f0b2-17c4-4abf-a5c8-8032482475f6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:15.980018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396f0b2-17c4-4abf-a5c8-8032482475f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:15.980475 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396f0b2-17c4-4abf-a5c8-8032482475f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:15.982103 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396f0b2-17c4-4abf-a5c8-8032482475f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:15.982730 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396f0b2-17c4-4abf-a5c8-8032482475f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:15.984473 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396f0b2-17c4-4abf-a5c8-8032482475f6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:16.004742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 358/706] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:16.008119 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc4102-a0ee-454a-85fc-1c13a5da15f0 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:16.008119 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc4102-a0ee-454a-85fc-1c13a5da15f0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:16.008119 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cc4102-a0ee-454a-85fc-1c13a5da15f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:16.024373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-02cc4102-a0ee-454a-85fc-1c13a5da15f0 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:16.025786 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-02cc4102-a0ee-454a-85fc-1c13a5da15f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:16.067614 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac202e5-7502-4fc3-9436-1fffcb71e10c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:16.098770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:16.099405 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:16.102759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 359/707] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:59:16.105263 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42df70-630d-4920-9b6f-9ea8a04f9dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:16.106065 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42df70-630d-4920-9b6f-9ea8a04f9dab None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:16.106353 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42df70-630d-4920-9b6f-9ea8a04f9dab None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:16.106660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42df70-630d-4920-9b6f-9ea8a04f9dab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:16.107064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc4102-a0ee-454a-85fc-1c13a5da15f0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:16.114391 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42df70-630d-4920-9b6f-9ea8a04f9dab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:16.118684 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42df70-630d-4920-9b6f-9ea8a04f9dab None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:16.132258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-0b42df70-630d-4920-9b6f-9ea8a04f9dab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:16.132258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:16.132258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:16.132258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:16.132258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:16.132258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:16.132258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:16.132258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:16.132258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:16.132258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:16.132258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:16.132258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:16.132258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:16.132258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:16.132258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:16.133180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:16.133180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:16.133180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:16.133180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:16.133180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:16.133180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:16.133180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:16.133180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:16.133180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:16.133180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:16.133180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:16.133180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:16.133180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:16.133180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:16.133180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:16.133180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:16.133180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:16.134078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:16.134078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:16.134078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:16.134078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:16.134078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:16.134078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:16.134078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:16.134078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:16.134078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 349/708] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:16.134078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42df70-630d-4920-9b6f-9ea8a04f9dab None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:16.223650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cc4102-a0ee-454a-85fc-1c13a5da15f0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:16.245190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42df70-630d-4920-9b6f-9ea8a04f9dab None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:16.261074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea390f85-bf9c-4867-a0bf-8be14cbfbefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:16.262071 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea390f85-bf9c-4867-a0bf-8be14cbfbefe None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:16.262684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea390f85-bf9c-4867-a0bf-8be14cbfbefe None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:16.263215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea390f85-bf9c-4867-a0bf-8be14cbfbefe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:16.264319 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea390f85-bf9c-4867-a0bf-8be14cbfbefe None admin] RBAC: Adding query filter params (role.id=f0f7e28cbe314921afdfb657b163436f, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=80b7b3ec60db4de19bbf7d08f111f470) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:16.264852 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea390f85-bf9c-4867-a0bf-8be14cbfbefe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:16.266617 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea390f85-bf9c-4867-a0bf-8be14cbfbefe None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:16.273964 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 360/709] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:59:16 2022] GET /identity/v3/role_assignments?user.id=80b7b3ec60db4de19bbf7d08f111f470&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=f0f7e28cbe314921afdfb657b163436f => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:16.275069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea390f85-bf9c-4867-a0bf-8be14cbfbefe None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:16.291845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fb6fa-26a4-4708-809d-bc9b55ea0e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:16.292670 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fb6fa-26a4-4708-809d-bc9b55ea0e54 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:16.292961 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fb6fa-26a4-4708-809d-bc9b55ea0e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:16.293245 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fb6fa-26a4-4708-809d-bc9b55ea0e54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:16.305597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66fb6fa-26a4-4708-809d-bc9b55ea0e54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:16.307161 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66fb6fa-26a4-4708-809d-bc9b55ea0e54 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:16.317749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:16.351957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-b66fb6fa-26a4-4708-809d-bc9b55ea0e54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:16.351957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:16.351957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:16.351957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:16.351957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:16.351957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:16.351957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:16.351957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:16.351957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:16.351957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:16.351957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:16.351957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:16.351957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:16.351957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:16.351957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:16.352981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:16.352981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:16.352981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:16.352981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:16.352981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:16.352981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:16.352981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:16.352981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:16.352981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:16.352981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:16.352981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:16.352981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:16.352981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:16.352981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:16.352981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:16.352981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:16.352981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:16.353938 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:16.353938 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:16.353938 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:16.353938 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:16.353938 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 350/710] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:16.363348 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fb6fa-26a4-4708-809d-bc9b55ea0e54 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:16.401640 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea390f85-bf9c-4867-a0bf-8be14cbfbefe None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:16.439414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2eb55-2130-497a-847b-79d339fa2490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:16.441658 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2eb55-2130-497a-847b-79d339fa2490 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:16.442168 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2eb55-2130-497a-847b-79d339fa2490 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:16.442670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2eb55-2130-497a-847b-79d339fa2490 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:16.443689 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2eb55-2130-497a-847b-79d339fa2490 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:16.444207 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2eb55-2130-497a-847b-79d339fa2490 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:16.445894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2eb55-2130-497a-847b-79d339fa2490 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:16.461513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 361/711] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:16.471809 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2eb55-2130-497a-847b-79d339fa2490 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:16.472495 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:16.571884 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fb6fa-26a4-4708-809d-bc9b55ea0e54 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:16.635207 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc779e5d-01a6-4c58-bd54-103d902d319c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:16.636539 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc779e5d-01a6-4c58-bd54-103d902d319c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:16.636898 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc779e5d-01a6-4c58-bd54-103d902d319c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:16.637249 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc779e5d-01a6-4c58-bd54-103d902d319c None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:16.638290 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc779e5d-01a6-4c58-bd54-103d902d319c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:16.638637 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc779e5d-01a6-4c58-bd54-103d902d319c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:16.640125 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:16.641889 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc779e5d-01a6-4c58-bd54-103d902d319c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:16.649656 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2eb55-2130-497a-847b-79d339fa2490 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:16.659437 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 351/712] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:16.661525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc8a09-e0bb-4912-b29d-25d814b66ba0 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:16.661966 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc8a09-e0bb-4912-b29d-25d814b66ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:16.662814 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc8a09-e0bb-4912-b29d-25d814b66ba0 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:16.666458 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 352/713] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:16.667401 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc8a09-e0bb-4912-b29d-25d814b66ba0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:16.700683 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfbec9d-0a5f-4e5c-ba1c-44f704dd8e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:16.701361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbec9d-0a5f-4e5c-ba1c-44f704dd8e10 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:16.701923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbec9d-0a5f-4e5c-ba1c-44f704dd8e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:16.702213 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbec9d-0a5f-4e5c-ba1c-44f704dd8e10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:16.703537 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfbec9d-0a5f-4e5c-ba1c-44f704dd8e10 None admin] RBAC: Adding query filter params (role.id=3073b5cac26f43089c9ed8c6a5102cff, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=6d8e1f1e3c7e42cc9fa6f82aaac55129) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:16.703914 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfbec9d-0a5f-4e5c-ba1c-44f704dd8e10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:16.706076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfbec9d-0a5f-4e5c-ba1c-44f704dd8e10 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:16.714810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 362/714] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:59:16 2022] GET /identity/v3/role_assignments?user.id=6d8e1f1e3c7e42cc9fa6f82aaac55129&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=3073b5cac26f43089c9ed8c6a5102cff => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:16.716252 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef25544-32a1-45ed-a206-b6003b5e602a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:16.716544 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef25544-32a1-45ed-a206-b6003b5e602a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:16.716825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef25544-32a1-45ed-a206-b6003b5e602a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:16.743591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4ef25544-32a1-45ed-a206-b6003b5e602a None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:16.745329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4ef25544-32a1-45ed-a206-b6003b5e602a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:16.787887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc8a09-e0bb-4912-b29d-25d814b66ba0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:16.790398 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:16.835742 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c95119-1abb-4123-b263-745b3520089a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:16.836840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c95119-1abb-4123-b263-745b3520089a None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:16.837387 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c95119-1abb-4123-b263-745b3520089a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:16.837876 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c95119-1abb-4123-b263-745b3520089a None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:16.839542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c95119-1abb-4123-b263-745b3520089a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:16.841959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c95119-1abb-4123-b263-745b3520089a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:16.847308 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 363/715] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:59:16.870552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef25544-32a1-45ed-a206-b6003b5e602a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:16.942145 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 353/716] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jun 24 03:59:16 2022] POST /identity/v3/users => generated 254 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:59:16.971547 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:16.989317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef25544-32a1-45ed-a206-b6003b5e602a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:17.033282 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e7c9d-00eb-4155-ac31-3757b1ab082f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:17.034899 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e7c9d-00eb-4155-ac31-3757b1ab082f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:17.035188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e7c9d-00eb-4155-ac31-3757b1ab082f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:17.035466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e7c9d-00eb-4155-ac31-3757b1ab082f None admin] PATH_INFO: `/v3/services/network` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:17.036160 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e7c9d-00eb-4155-ac31-3757b1ab082f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:17.037583 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e7c9d-00eb-4155-ac31-3757b1ab082f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:17.057981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-be8e7c9d-00eb-4155-ac31-3757b1ab082f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 03:59:17.057981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:17.057981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:17.057981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:17.057981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:17.057981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:17.057981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:17.057981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:17.057981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 03:59:17.057981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:17.057981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:17.057981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:17.057981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:17.057981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:17.057981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:17.057981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:17.057981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:17.059203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:17.059203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:17.059203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:17.059203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:17.059203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:17.059203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:17.059203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:17.059203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:17.059203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:17.059203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:17.059203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:17.059203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:17.059203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:17.059203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:17.059203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:17.059203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:17.060604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:17.060604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:17.060604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:17.060604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:17.060604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:17.060604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:17.060604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:17.060604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:17.060604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:17.060604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:17.060604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:17.060604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:17.060604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 03:59:17.060604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:17.060604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 364/717] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:16 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:17.069633 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c95119-1abb-4123-b263-745b3520089a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:17.202898 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c95119-1abb-4123-b263-745b3520089a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:17.239691 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec9449-4f28-499c-a370-b2fd89f0dec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:17.240369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec9449-4f28-499c-a370-b2fd89f0dec3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:17.240726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec9449-4f28-499c-a370-b2fd89f0dec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:17.240997 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec9449-4f28-499c-a370-b2fd89f0dec3 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:17.241716 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec9449-4f28-499c-a370-b2fd89f0dec3 None admin] RBAC: Adding query filter params (name=network) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:17.242028 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec9449-4f28-499c-a370-b2fd89f0dec3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:17.243448 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec9449-4f28-499c-a370-b2fd89f0dec3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:17.248632 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 354/718] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jun 24 03:59:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:17.254974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e7c9d-00eb-4155-ac31-3757b1ab082f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:17.366975 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e7c9d-00eb-4155-ac31-3757b1ab082f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:17.419720 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696baec0-a195-4b07-8103-8c7102a91c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:17.420386 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696baec0-a195-4b07-8103-8c7102a91c5e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:17.420676 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696baec0-a195-4b07-8103-8c7102a91c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:17.420957 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696baec0-a195-4b07-8103-8c7102a91c5e None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:17.421674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696baec0-a195-4b07-8103-8c7102a91c5e None admin] RBAC: Adding query filter params (type=network) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:17.422039 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696baec0-a195-4b07-8103-8c7102a91c5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:17.423464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696baec0-a195-4b07-8103-8c7102a91c5e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:17.431651 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 365/719] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jun 24 03:59:17 2022] GET /identity/v3/services?type=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:18.266010 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4675a-cf8b-430b-9c4e-3cc74c0e31e8 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:18.266541 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4675a-cf8b-430b-9c4e-3cc74c0e31e8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:18.266643 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4675a-cf8b-430b-9c4e-3cc74c0e31e8 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:18.268260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 355/720] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:18.288376 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa43c3-02bc-4728-9c0f-58bf27c356e6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:18.288857 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa43c3-02bc-4728-9c0f-58bf27c356e6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:18.288934 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa43c3-02bc-4728-9c0f-58bf27c356e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:18.315450 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5aaa43c3-02bc-4728-9c0f-58bf27c356e6 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:18.316584 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5aaa43c3-02bc-4728-9c0f-58bf27c356e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:18.326652 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:18.404707 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 366/721] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:59:18.429069 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4675a-cf8b-430b-9c4e-3cc74c0e31e8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:18.475023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:18.584320 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4675a-cf8b-430b-9c4e-3cc74c0e31e8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:18.638129 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0f7df1-6a68-4cf1-8702-a7e361dc10e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:18.638836 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f7df1-6a68-4cf1-8702-a7e361dc10e2 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:18.639124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f7df1-6a68-4cf1-8702-a7e361dc10e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:18.639407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0f7df1-6a68-4cf1-8702-a7e361dc10e2 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:18.640151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f7df1-6a68-4cf1-8702-a7e361dc10e2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:18.641545 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0f7df1-6a68-4cf1-8702-a7e361dc10e2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:18.756330 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 356/722] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jun 24 03:59:18 2022] POST /identity/v3/services => generated 251 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:59:18.822724 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369e794-4e23-4994-9078-8d742885d5c7 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:18.823083 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369e794-4e23-4994-9078-8d742885d5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:18.823292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d369e794-4e23-4994-9078-8d742885d5c7 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:18.825069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 367/723] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:18.832199 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffeb9d-3842-4b5e-990b-514c46e9f0d3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:18.832846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffeb9d-3842-4b5e-990b-514c46e9f0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:18.833329 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffeb9d-3842-4b5e-990b-514c46e9f0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:18.864136 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-50ffeb9d-3842-4b5e-990b-514c46e9f0d3 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:18.865717 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-50ffeb9d-3842-4b5e-990b-514c46e9f0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:19.015670 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 357/724] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:59:19.058194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369e794-4e23-4994-9078-8d742885d5c7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:19.195035 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d369e794-4e23-4994-9078-8d742885d5c7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:19.249933 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd3749-69ea-4f42-8e31-2c1cc9b46800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:19.250812 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd3749-69ea-4f42-8e31-2c1cc9b46800 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:19.251106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd3749-69ea-4f42-8e31-2c1cc9b46800 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:19.251389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd3749-69ea-4f42-8e31-2c1cc9b46800 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:19.266942 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd3749-69ea-4f42-8e31-2c1cc9b46800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:19.275814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-fabd3749-69ea-4f42-8e31-2c1cc9b46800 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:59:19.275814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:19.275814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:19.275814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:19.275814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:19.275814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:19.275814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:19.275814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:19.275814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:19.275814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:19.275814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:19.275814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:19.275814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:19.275814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:19.275814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:19.275814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:19.277077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:19.277077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:19.277077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:19.277077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:19.277077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:19.277077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:19.277077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:19.277077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:19.277077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:19.277077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:19.277077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:19.277077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:19.277077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:19.277077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:19.277077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:19.277077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:19.278620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:19.278620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:19.278620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:19.278620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:19.278620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:19.278620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:19.278620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:19.278620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:19.278620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:19.278620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:19.278620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:59:19.278620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:19.278620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 368/725] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:59:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:19.289165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffeb9d-3842-4b5e-990b-514c46e9f0d3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:19.308098 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:19.432130 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffeb9d-3842-4b5e-990b-514c46e9f0d3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:19.475135 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0062d-fa67-481d-84c9-067000636797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:19.475734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0062d-fa67-481d-84c9-067000636797 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:19.476025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0062d-fa67-481d-84c9-067000636797 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:19.476322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0062d-fa67-481d-84c9-067000636797 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:19.477070 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0062d-fa67-481d-84c9-067000636797 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:19.477521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0062d-fa67-481d-84c9-067000636797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:19.479783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0062d-fa67-481d-84c9-067000636797 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:19.491370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 358/726] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:59:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:19.499707 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd3749-69ea-4f42-8e31-2c1cc9b46800 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:19.503781 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:19.607033 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd3749-69ea-4f42-8e31-2c1cc9b46800 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:19.644318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaffdb0-1143-4a5f-92d2-cd7a373d8673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:19.645121 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaffdb0-1143-4a5f-92d2-cd7a373d8673 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:19.645412 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaffdb0-1143-4a5f-92d2-cd7a373d8673 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:19.645695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaffdb0-1143-4a5f-92d2-cd7a373d8673 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:19.657677 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaffdb0-1143-4a5f-92d2-cd7a373d8673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:19.659516 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaffdb0-1143-4a5f-92d2-cd7a373d8673 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:19.677682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-8eaffdb0-1143-4a5f-92d2-cd7a373d8673 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:19.677682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:19.677682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:19.677682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:19.677682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:19.677682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:19.677682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:19.677682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:19.677682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:19.677682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:19.677682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:19.677682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:19.677682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:19.677682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:19.677682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:19.679784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:19.679784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:19.679784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:19.679784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:19.679784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:19.679784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:19.679784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:19.679784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:19.679784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:19.679784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:19.679784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:19.679784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:19.679784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:19.679784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:19.679784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:19.679784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:19.679784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:19.681913 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:19.681913 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:19.681913 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:19.681913 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:19.681913 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 369/727] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:19.688062 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0062d-fa67-481d-84c9-067000636797 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:19.779290 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:19.825272 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0062d-fa67-481d-84c9-067000636797 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:19.833079 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0056f8-856e-414a-bb62-f0d40ba1cdc6 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:19.833079 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0056f8-856e-414a-bb62-f0d40ba1cdc6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:19.833079 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0056f8-856e-414a-bb62-f0d40ba1cdc6 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:19.833880 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 370/728] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:19.839237 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bb1b7-bcfa-4fa2-97ae-2cdb651009d9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:19.839772 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bb1b7-bcfa-4fa2-97ae-2cdb651009d9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:19.840246 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bb1b7-bcfa-4fa2-97ae-2cdb651009d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:19.857209 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d28bb1b7-bcfa-4fa2-97ae-2cdb651009d9 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:19.858821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d28bb1b7-bcfa-4fa2-97ae-2cdb651009d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:19.884622 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de5d28-6c8a-470c-97c1-7e8536629490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:19.885653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de5d28-6c8a-470c-97c1-7e8536629490 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:19.886134 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de5d28-6c8a-470c-97c1-7e8536629490 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:19.886631 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de5d28-6c8a-470c-97c1-7e8536629490 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:19.887635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de5d28-6c8a-470c-97c1-7e8536629490 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:19.888149 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de5d28-6c8a-470c-97c1-7e8536629490 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:19.889775 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de5d28-6c8a-470c-97c1-7e8536629490 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:19.903674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 359/729] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:19.909537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99836d8-6c68-4e3f-9ac9-28ad1a21c13f None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:19.909537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99836d8-6c68-4e3f-9ac9-28ad1a21c13f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:19.909537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99836d8-6c68-4e3f-9ac9-28ad1a21c13f None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:19.914454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 360/730] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:19.917068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99836d8-6c68-4e3f-9ac9-28ad1a21c13f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:19.936108 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:19.960220 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 371/731] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 03:59:19.966523 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d910d-b59c-48cc-ae6f-7a1ddc358ea3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:19.966523 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d910d-b59c-48cc-ae6f-7a1ddc358ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:19.966523 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d910d-b59c-48cc-ae6f-7a1ddc358ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:19.984141 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-942d910d-b59c-48cc-ae6f-7a1ddc358ea3 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:19.985701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-942d910d-b59c-48cc-ae6f-7a1ddc358ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:20.062303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99836d8-6c68-4e3f-9ac9-28ad1a21c13f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:20.074890 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 372/732] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:19 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:59:20.075784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d910d-b59c-48cc-ae6f-7a1ddc358ea3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:20.112434 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:20.120205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1e624-6dcf-4f57-ac0b-3cfa3c86e449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:20.124222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1e624-6dcf-4f57-ac0b-3cfa3c86e449 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:20.124222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1e624-6dcf-4f57-ac0b-3cfa3c86e449 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:20.124222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1e624-6dcf-4f57-ac0b-3cfa3c86e449 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:20.135001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1e624-6dcf-4f57-ac0b-3cfa3c86e449 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:20.136475 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1e624-6dcf-4f57-ac0b-3cfa3c86e449 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:20.156738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-0cb1e624-6dcf-4f57-ac0b-3cfa3c86e449 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 03:59:20.156738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:20.156738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:20.156738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:20.156738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:20.156738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:20.156738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:20.156738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:20.156738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:20.156738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:20.156738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:20.156738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:20.156738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:20.156738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:20.156738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:20.156738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:20.157942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:20.157942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:20.157942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:20.157942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:20.157942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:20.157942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:20.157942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:20.157942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:20.157942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:20.157942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:20.157942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:20.157942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:20.157942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:20.157942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:20.157942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:20.157942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:20.157942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:20.157942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:20.159321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:20.159321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:20.159321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:20.159321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:20.159321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:20.159321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:20.159321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:20.159321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:20.159321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 03:59:20.159321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:20.160254 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 361/733] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Jun 24 03:59:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:20.160757 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1e624-6dcf-4f57-ac0b-3cfa3c86e449 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:20.207226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d910d-b59c-48cc-ae6f-7a1ddc358ea3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:20.224780 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:20.244825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca0e71-f68f-4a87-8648-fca4d580a253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:20.246009 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca0e71-f68f-4a87-8648-fca4d580a253 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:20.246572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca0e71-f68f-4a87-8648-fca4d580a253 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:20.247049 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ca0e71-f68f-4a87-8648-fca4d580a253 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:20.259480 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ca0e71-f68f-4a87-8648-fca4d580a253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:20.268031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-35ca0e71-f68f-4a87-8648-fca4d580a253 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:59:20.268031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:20.268031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:20.268031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:20.268031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:20.268031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:20.268031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:20.268031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:20.268031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:20.268031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:20.268031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:20.268031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:20.268031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:20.268031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:20.268031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:20.268031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:20.269215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:20.269215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:20.269215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:20.269215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:20.269215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:20.269215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:20.269215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:20.269215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:20.269215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:20.269215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:20.269215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:20.269215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:20.269215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:20.269215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:20.269215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:20.269215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:20.270076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:20.270076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:20.270076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:20.270076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:20.270076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:20.270076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:20.270076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:20.270076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:20.270076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:20.270076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:20.270076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:59:20.270076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:20.274184 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 373/734] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:59:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:20.275001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca0e71-f68f-4a87-8648-fca4d580a253 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:20.291442 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1e624-6dcf-4f57-ac0b-3cfa3c86e449 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:20.328988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9dbaa-057b-4f18-8bd6-a9765fadd8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:20.329709 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9dbaa-057b-4f18-8bd6-a9765fadd8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:20.330554 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9dbaa-057b-4f18-8bd6-a9765fadd8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:20.330837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9dbaa-057b-4f18-8bd6-a9765fadd8e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:20.344364 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9dbaa-057b-4f18-8bd6-a9765fadd8e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:20.352871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-96a9dbaa-057b-4f18-8bd6-a9765fadd8e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:20.352871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:20.352871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:20.352871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:20.352871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:20.352871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:20.352871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:20.352871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:20.352871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:20.352871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:20.352871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:20.352871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:20.352871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:20.352871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:20.352871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:20.352871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:20.353849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:20.353849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:20.353849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:20.353849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:20.353849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:20.353849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:20.353849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:20.353849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:20.353849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:20.353849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:20.353849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:20.353849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:20.353849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:20.353849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:20.353849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:20.353849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:20.359953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:20.359953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:20.359953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:20.359953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:20.359953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:20.359953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:20.359953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:20.359953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:20.359953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:20.359953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:20.359953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:20.359953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:20.359953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 362/735] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:59:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:20.359953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db5884-242f-4024-9941-ffd90f71f16c None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:20.360732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db5884-242f-4024-9941-ffd90f71f16c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:20.360732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db5884-242f-4024-9941-ffd90f71f16c None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:20.360732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 363/736] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:20.360732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db5884-242f-4024-9941-ffd90f71f16c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:20.388911 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ca0e71-f68f-4a87-8648-fca4d580a253 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:20.414407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:20.427102 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d10ba1-ed67-4bba-9f13-8eae76bf5cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:20.428038 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d10ba1-ed67-4bba-9f13-8eae76bf5cfe None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:20.428406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d10ba1-ed67-4bba-9f13-8eae76bf5cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:20.428692 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d10ba1-ed67-4bba-9f13-8eae76bf5cfe None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:20.429535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d10ba1-ed67-4bba-9f13-8eae76bf5cfe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:20.429852 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d10ba1-ed67-4bba-9f13-8eae76bf5cfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:20.431365 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d10ba1-ed67-4bba-9f13-8eae76bf5cfe None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:20.450335 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 374/737] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Jun 24 03:59:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:20.451831 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904f78c-ce66-4390-baea-9731298c73bb None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:20.452284 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904f78c-ce66-4390-baea-9731298c73bb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:20.452754 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a904f78c-ce66-4390-baea-9731298c73bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:20.469794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a904f78c-ce66-4390-baea-9731298c73bb None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:20.471514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a904f78c-ce66-4390-baea-9731298c73bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:20.472899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db5884-242f-4024-9941-ffd90f71f16c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:20.500794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:20.510958 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7565d7-90e2-479e-bd7a-c026a32d3341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:20.511624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7565d7-90e2-479e-bd7a-c026a32d3341 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:20.511906 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7565d7-90e2-479e-bd7a-c026a32d3341 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:20.512183 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7565d7-90e2-479e-bd7a-c026a32d3341 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:20.512921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7565d7-90e2-479e-bd7a-c026a32d3341 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:20.513273 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7565d7-90e2-479e-bd7a-c026a32d3341 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:20.514756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7565d7-90e2-479e-bd7a-c026a32d3341 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:20.524590 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 364/738] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:59:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:20.525396 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7565d7-90e2-479e-bd7a-c026a32d3341 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:20.548603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 375/739] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:20 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 03:59:20.550437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904f78c-ce66-4390-baea-9731298c73bb None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:20.630248 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7565d7-90e2-479e-bd7a-c026a32d3341 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:20.644065 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:20.665795 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a904f78c-ce66-4390-baea-9731298c73bb None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:20.665966 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38664c5-4cc1-4d80-9099-2a6c69384395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:20.666609 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38664c5-4cc1-4d80-9099-2a6c69384395 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:20.666893 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38664c5-4cc1-4d80-9099-2a6c69384395 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:20.667167 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38664c5-4cc1-4d80-9099-2a6c69384395 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:20.667955 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38664c5-4cc1-4d80-9099-2a6c69384395 None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:20.668255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38664c5-4cc1-4d80-9099-2a6c69384395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:20.669624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38664c5-4cc1-4d80-9099-2a6c69384395 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:20.678866 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 365/740] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 03:59:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:20.679550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38664c5-4cc1-4d80-9099-2a6c69384395 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:20.794221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38664c5-4cc1-4d80-9099-2a6c69384395 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:20.818440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4bdd1-3bc0-4469-89e3-e79c5c83e751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:20.819238 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4bdd1-3bc0-4469-89e3-e79c5c83e751 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:20.819522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4bdd1-3bc0-4469-89e3-e79c5c83e751 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:20.819798 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4bdd1-3bc0-4469-89e3-e79c5c83e751 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:20.829785 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a9f0f-854c-40d7-8693-e2b7a00203bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:20.830600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4bdd1-3bc0-4469-89e3-e79c5c83e751 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:20.831204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a9f0f-854c-40d7-8693-e2b7a00203bd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:20.831483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a9f0f-854c-40d7-8693-e2b7a00203bd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:20.831754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a9f0f-854c-40d7-8693-e2b7a00203bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:20.832018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4bdd1-3bc0-4469-89e3-e79c5c83e751 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:20.843367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a9f0f-854c-40d7-8693-e2b7a00203bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:20.844797 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a9f0f-854c-40d7-8693-e2b7a00203bd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:20.850321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-72d4bdd1-3bc0-4469-89e3-e79c5c83e751 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:20.850321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:20.850321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:20.850321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:20.850321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:20.850321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:20.850321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:20.850321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:20.850321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:20.850321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:20.850321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:20.850321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:20.850321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:20.850321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:20.850321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:20.851315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:20.851315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:20.851315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:20.851315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:20.851315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:20.851315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:20.851315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:20.851315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:20.851315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:20.851315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:20.851315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:20.851315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:20.851315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:20.851315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:20.851315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:20.851315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:20.851315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:20.852244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:20.852244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:20.852244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:20.852244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:20.854778 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 376/741] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:20.855617 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4bdd1-3bc0-4469-89e3-e79c5c83e751 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:20.865619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-6b5a9f0f-854c-40d7-8693-e2b7a00203bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:20.865619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:20.865619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:20.865619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:20.865619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:20.865619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:20.865619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:20.865619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:20.865619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:20.865619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:20.865619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:20.865619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:20.865619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:20.865619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:20.865619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:20.866589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:20.866589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:20.866589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:20.866589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:20.866589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:20.866589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:20.866589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:20.866589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:20.866589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:20.866589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:20.866589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:20.866589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:20.866589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:20.866589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:20.866589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:20.866589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:20.866589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:20.867485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:20.867485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:20.867485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:20.867485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:20.870675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 366/742] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:20.872096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a9f0f-854c-40d7-8693-e2b7a00203bd None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:20.878541 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:20.904990 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:20.969299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4bdd1-3bc0-4469-89e3-e79c5c83e751 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:20.984887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a9f0f-854c-40d7-8693-e2b7a00203bd None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:21.006346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64ac01-b673-498b-9913-19dc20e10b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:21.007043 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64ac01-b673-498b-9913-19dc20e10b8e None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:21.007339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64ac01-b673-498b-9913-19dc20e10b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:21.007620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64ac01-b673-498b-9913-19dc20e10b8e None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:21.008345 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64ac01-b673-498b-9913-19dc20e10b8e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:21.009726 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64ac01-b673-498b-9913-19dc20e10b8e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:21.020202 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f356a2-aadf-4251-9bf5-883ab6e2875e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:21.020988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f356a2-aadf-4251-9bf5-883ab6e2875e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:21.021275 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f356a2-aadf-4251-9bf5-883ab6e2875e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:21.021554 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f356a2-aadf-4251-9bf5-883ab6e2875e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:21.032378 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f356a2-aadf-4251-9bf5-883ab6e2875e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:21.034440 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f356a2-aadf-4251-9bf5-883ab6e2875e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:21.052518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-08f356a2-aadf-4251-9bf5-883ab6e2875e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:21.052518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:21.052518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:21.052518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:21.052518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:21.052518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:21.052518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:21.052518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:21.052518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:21.052518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:21.052518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:21.052518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:21.052518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:21.052518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:21.052518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:21.053534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:21.053534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:21.053534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:21.053534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:21.053534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:21.053534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:21.053534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:21.053534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:21.053534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:21.053534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:21.053534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:21.053534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:21.053534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:21.053534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:21.053534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:21.053534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:21.053534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:21.054469 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:21.054469 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:21.054469 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:21.054469 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:21.054469 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 367/743] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:21.054756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f356a2-aadf-4251-9bf5-883ab6e2875e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:21.073065 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 377/744] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jun 24 03:59:20 2022] POST /identity/v3/services => generated 240 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:59:21.073982 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64ac01-b673-498b-9913-19dc20e10b8e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:21.168821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:21.175683 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:21.201637 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f356a2-aadf-4251-9bf5-883ab6e2875e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:21.212737 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64ac01-b673-498b-9913-19dc20e10b8e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:21.245962 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdae47-ba4b-4b3b-a23f-f1524598d485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:21.246725 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdae47-ba4b-4b3b-a23f-f1524598d485 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:21.247116 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdae47-ba4b-4b3b-a23f-f1524598d485 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:21.247464 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdae47-ba4b-4b3b-a23f-f1524598d485 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:21.248215 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdae47-ba4b-4b3b-a23f-f1524598d485 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:21.248534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdae47-ba4b-4b3b-a23f-f1524598d485 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:21.249910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cdae47-ba4b-4b3b-a23f-f1524598d485 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:21.258595 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c73cde-f183-43a6-a53d-f3859e115f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:21.259273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c73cde-f183-43a6-a53d-f3859e115f87 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:21.259564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c73cde-f183-43a6-a53d-f3859e115f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:21.259844 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c73cde-f183-43a6-a53d-f3859e115f87 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:21.260581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c73cde-f183-43a6-a53d-f3859e115f87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:21.260896 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c73cde-f183-43a6-a53d-f3859e115f87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:21.262284 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c73cde-f183-43a6-a53d-f3859e115f87 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:21.264358 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 368/745] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:21.265121 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdae47-ba4b-4b3b-a23f-f1524598d485 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:21.279620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 378/746] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:21.280479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c73cde-f183-43a6-a53d-f3859e115f87 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:21.380105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cdae47-ba4b-4b3b-a23f-f1524598d485 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:21.387116 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c73cde-f183-43a6-a53d-f3859e115f87 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:21.417313 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:21.417712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c32b1b-7f5e-4fa9-9017-90c71e791d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:21.419014 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c32b1b-7f5e-4fa9-9017-90c71e791d0d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:21.419503 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c32b1b-7f5e-4fa9-9017-90c71e791d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:21.419851 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c32b1b-7f5e-4fa9-9017-90c71e791d0d None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:21.420581 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c32b1b-7f5e-4fa9-9017-90c71e791d0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:21.420894 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c32b1b-7f5e-4fa9-9017-90c71e791d0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:21.422271 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c32b1b-7f5e-4fa9-9017-90c71e791d0d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:21.422694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1423f98a-b234-4d52-b904-7233ad2b92ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:21.423484 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423f98a-b234-4d52-b904-7233ad2b92ea None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:21.423958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423f98a-b234-4d52-b904-7233ad2b92ea None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:21.424210 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1423f98a-b234-4d52-b904-7233ad2b92ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:21.430126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:21.431366 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1423f98a-b234-4d52-b904-7233ad2b92ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:21.432798 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1423f98a-b234-4d52-b904-7233ad2b92ea None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:21.437954 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 369/747] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:21.438816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c32b1b-7f5e-4fa9-9017-90c71e791d0d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:21.451796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-1423f98a-b234-4d52-b904-7233ad2b92ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:21.451796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:21.451796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:21.451796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:21.451796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:21.451796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:21.451796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:21.451796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:21.451796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:21.451796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:21.451796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:21.451796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:21.451796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:21.451796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:21.451796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:21.452927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:21.452927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:21.452927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:21.452927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:21.452927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:21.452927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:21.452927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:21.452927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:21.452927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:21.452927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:21.452927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:21.452927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:21.452927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:21.452927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:21.452927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:21.452927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:21.452927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:21.454007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:21.454007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:21.454007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:21.454007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:21.454007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:21.454007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:21.454007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:21.454007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:21.457126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 379/748] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:21.458618 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda71dca-07aa-4be2-8388-486584e46961 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:21.458907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda71dca-07aa-4be2-8388-486584e46961 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:21.464313 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda71dca-07aa-4be2-8388-486584e46961 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:21.464313 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 380/749] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:21.464313 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda71dca-07aa-4be2-8388-486584e46961 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:21.566736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c32b1b-7f5e-4fa9-9017-90c71e791d0d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:21.570835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda71dca-07aa-4be2-8388-486584e46961 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:21.603402 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5acd8-1342-4bd3-bd11-33a850b303c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:21.604524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5acd8-1342-4bd3-bd11-33a850b303c5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:21.604708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d909c03-37ae-4385-b001-668107a3446e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:21.605174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5acd8-1342-4bd3-bd11-33a850b303c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:21.605516 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d909c03-37ae-4385-b001-668107a3446e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:21.605648 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5acd8-1342-4bd3-bd11-33a850b303c5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:21.605808 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d909c03-37ae-4385-b001-668107a3446e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:21.606067 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d909c03-37ae-4385-b001-668107a3446e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:21.614607 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d909c03-37ae-4385-b001-668107a3446e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:21.616065 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d909c03-37ae-4385-b001-668107a3446e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:21.619235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5acd8-1342-4bd3-bd11-33a850b303c5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:21.620877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5acd8-1342-4bd3-bd11-33a850b303c5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:21.632442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-6d909c03-37ae-4385-b001-668107a3446e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 03:59:21.632442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:21.632442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:21.632442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:21.632442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:21.632442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:21.632442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:21.632442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:21.632442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:21.632442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:21.632442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:21.632442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:21.632442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:21.632442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:21.632442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:21.632442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:21.633446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:21.633446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:21.633446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:21.633446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:21.633446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:21.633446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:21.633446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:21.633446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:21.633446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:21.633446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:21.633446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:21.633446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:21.633446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:21.633446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:21.633446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:21.633446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:21.633446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:21.633446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:21.634460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:21.634460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:21.634460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:21.634460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:21.634460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:21.634460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:21.634460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:21.634460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:21.634460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 03:59:21.634460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:21.637055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-75f5acd8-1342-4bd3-bd11-33a850b303c5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 03:59:21.637055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:21.637055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:21.637055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:21.637055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:21.637055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:21.637055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:21.637055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:21.637055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:21.637055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:21.637055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:21.637055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:21.637055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:21.637055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:21.637055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:21.637055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:21.637943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:21.637943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:21.637943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:21.637943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:21.637943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:21.637943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:21.637943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:21.637943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:21.637943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:21.637943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:21.637943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:21.637943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:21.637943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:21.637943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:21.637943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:21.637943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:21.637943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:21.637943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:21.638934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:21.638934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:21.638934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:21.638934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:21.638934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:21.638934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:21.638934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:21.638934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:21.638934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 03:59:21.638934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:21.639479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 381/750] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jun 24 03:59:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:21.639549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d909c03-37ae-4385-b001-668107a3446e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:21.642511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 370/751] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:59:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:21.643901 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a31f3-dc3e-45b4-93f1-9abe82e79457 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:21.644201 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a31f3-dc3e-45b4-93f1-9abe82e79457 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:21.644487 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a31f3-dc3e-45b4-93f1-9abe82e79457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:21.665257 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-442a31f3-dc3e-45b4-93f1-9abe82e79457 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:21.666770 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-442a31f3-dc3e-45b4-93f1-9abe82e79457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:21.696443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:21.726341 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:21.742417 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 371/752] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:21.742804 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a31f3-dc3e-45b4-93f1-9abe82e79457 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:21.752925 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d909c03-37ae-4385-b001-668107a3446e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:21.789491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5e734-4b43-46ec-a5b5-aa276fc2e04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:21.790253 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5e734-4b43-46ec-a5b5-aa276fc2e04e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:21.790637 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5e734-4b43-46ec-a5b5-aa276fc2e04e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:21.790915 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5e734-4b43-46ec-a5b5-aa276fc2e04e None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:21.791716 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5e734-4b43-46ec-a5b5-aa276fc2e04e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:21.792028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5e734-4b43-46ec-a5b5-aa276fc2e04e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:21.793462 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5e734-4b43-46ec-a5b5-aa276fc2e04e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:21.806138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 382/753] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:21.806974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5e734-4b43-46ec-a5b5-aa276fc2e04e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:21.867449 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a31f3-dc3e-45b4-93f1-9abe82e79457 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:21.905969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4a06b-c44f-4c0e-babe-b66ef60ba726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:21.907066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4a06b-c44f-4c0e-babe-b66ef60ba726 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:21.907502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4a06b-c44f-4c0e-babe-b66ef60ba726 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:21.908457 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4a06b-c44f-4c0e-babe-b66ef60ba726 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:21.909693 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4a06b-c44f-4c0e-babe-b66ef60ba726 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:21.910173 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4a06b-c44f-4c0e-babe-b66ef60ba726 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:21.911935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4a06b-c44f-4c0e-babe-b66ef60ba726 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:21.932045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 372/754] 10.222.0.16 () {56 vars in 1149 bytes} [Fri Jun 24 03:59:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:21.932269 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5e734-4b43-46ec-a5b5-aa276fc2e04e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:21.933166 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4a06b-c44f-4c0e-babe-b66ef60ba726 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:21.948820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:21.970154 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2fce1-385d-441f-96f0-7671b9f3cc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:21.971462 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2fce1-385d-441f-96f0-7671b9f3cc49 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:21.971743 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2fce1-385d-441f-96f0-7671b9f3cc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:21.972014 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2fce1-385d-441f-96f0-7671b9f3cc49 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:21.972988 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2fce1-385d-441f-96f0-7671b9f3cc49 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:21.973303 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2fce1-385d-441f-96f0-7671b9f3cc49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:21.974785 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2fce1-385d-441f-96f0-7671b9f3cc49 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:21.985420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:21.993411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 383/755] 10.222.0.16 () {56 vars in 1135 bytes} [Fri Jun 24 03:59:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:21.994971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2fce1-385d-441f-96f0-7671b9f3cc49 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:22.065223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4a06b-c44f-4c0e-babe-b66ef60ba726 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:22.107672 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2fce1-385d-441f-96f0-7671b9f3cc49 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:22.109339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0947d-e253-43ee-af8f-43bba47102ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:22.110310 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0947d-e253-43ee-af8f-43bba47102ee None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:22.110617 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0947d-e253-43ee-af8f-43bba47102ee None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:22.110894 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0947d-e253-43ee-af8f-43bba47102ee None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:22.112314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0947d-e253-43ee-af8f-43bba47102ee None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:22.113587 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0947d-e253-43ee-af8f-43bba47102ee None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:22.126521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-68c0947d-e253-43ee-af8f-43bba47102ee None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 03:59:22.126521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:22.126521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:22.126521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:22.126521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:22.126521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:22.126521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:22.126521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:22.126521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 03:59:22.126521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:22.126521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:22.126521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:22.126521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:22.126521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:22.126521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:22.126521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:22.128002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:22.128002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:22.128002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:22.128002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:22.128002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:22.128002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:22.128002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:22.128002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:22.128002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:22.128002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:22.128002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:22.128002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:22.128002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:22.128002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:22.128002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:22.128002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:22.128831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:22.128831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:22.128831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:22.128831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:22.128831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:22.128831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:22.128831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:22.128831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:22.128831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:22.128831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:22.128831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:22.128831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:22.128831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:22.128831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 03:59:22.128831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:22.133246 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 373/756] 10.222.0.16 () {56 vars in 1129 bytes} [Fri Jun 24 03:59:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:22.134132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0947d-e253-43ee-af8f-43bba47102ee None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:22.155824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c297024-d370-4dcc-89e7-09962b781beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:22.156817 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c297024-d370-4dcc-89e7-09962b781beb None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:22.157163 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c297024-d370-4dcc-89e7-09962b781beb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:22.157504 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c297024-d370-4dcc-89e7-09962b781beb None admin] PATH_INFO: `/v3/projects/68d376986e854a4bb334676572340b4c/users/80b7b3ec60db4de19bbf7d08f111f470/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:22.160964 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c297024-d370-4dcc-89e7-09962b781beb None admin] RBAC: Authorizing `identity:create_grant(project_id=68d376986e854a4bb334676572340b4c, user_id=80b7b3ec60db4de19bbf7d08f111f470, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:22.162516 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c297024-d370-4dcc-89e7-09962b781beb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:22.241399 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0947d-e253-43ee-af8f-43bba47102ee None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:22.254660 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 384/757] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 03:59:21 2022] PUT /identity/v3/projects/68d376986e854a4bb334676572340b4c/users/80b7b3ec60db4de19bbf7d08f111f470/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:59:22.254660 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c297024-d370-4dcc-89e7-09962b781beb None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:22.261791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:22.304472 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76efc8c3-5298-43a2-b018-19a008a39b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:22.305151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efc8c3-5298-43a2-b018-19a008a39b18 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:22.305442 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efc8c3-5298-43a2-b018-19a008a39b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:22.305723 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76efc8c3-5298-43a2-b018-19a008a39b18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:22.338031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:22.439764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76efc8c3-5298-43a2-b018-19a008a39b18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:22.442559 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76efc8c3-5298-43a2-b018-19a008a39b18 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:22.467057 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c297024-d370-4dcc-89e7-09962b781beb None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:22.507108 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d4d93-209a-4670-a9f9-8374d16cbc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:22.507678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d4d93-209a-4670-a9f9-8374d16cbc99 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:22.507961 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d4d93-209a-4670-a9f9-8374d16cbc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:22.508237 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288d4d93-209a-4670-a9f9-8374d16cbc99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:22.581920 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d4d93-209a-4670-a9f9-8374d16cbc99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:22.583482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288d4d93-209a-4670-a9f9-8374d16cbc99 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:22.593056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 374/758] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:59:22 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 03:59:22.594328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76efc8c3-5298-43a2-b018-19a008a39b18 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:22.609386 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:22.618879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:22.704229 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 385/759] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:59:22 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 03:59:22.705861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d4d93-209a-4670-a9f9-8374d16cbc99 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:22.726064 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76efc8c3-5298-43a2-b018-19a008a39b18 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:22.765934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226b16a-85f6-4dfe-a5e9-d8f426d07a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:22.766578 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226b16a-85f6-4dfe-a5e9-d8f426d07a61 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:22.766868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226b16a-85f6-4dfe-a5e9-d8f426d07a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:22.767150 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3226b16a-85f6-4dfe-a5e9-d8f426d07a61 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:22.769412 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226b16a-85f6-4dfe-a5e9-d8f426d07a61 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:22.769729 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226b16a-85f6-4dfe-a5e9-d8f426d07a61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:22.771393 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3226b16a-85f6-4dfe-a5e9-d8f426d07a61 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:22.777861 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 375/760] 10.222.0.16 () {56 vars in 1124 bytes} [Fri Jun 24 03:59:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:22.779051 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226b16a-85f6-4dfe-a5e9-d8f426d07a61 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:22.827814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288d4d93-209a-4670-a9f9-8374d16cbc99 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:22.857641 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:22.864544 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e5e278-a996-43a0-8f9f-6cc41f1d5150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:22.871279 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5e278-a996-43a0-8f9f-6cc41f1d5150 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:22.871279 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5e278-a996-43a0-8f9f-6cc41f1d5150 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:22.871279 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e5e278-a996-43a0-8f9f-6cc41f1d5150 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:22.871536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:22.879574 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e5e278-a996-43a0-8f9f-6cc41f1d5150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:22.881267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e5e278-a996-43a0-8f9f-6cc41f1d5150 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:22.895391 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3226b16a-85f6-4dfe-a5e9-d8f426d07a61 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:22.903928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-69e5e278-a996-43a0-8f9f-6cc41f1d5150 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:22.903928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:22.903928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:22.903928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:22.903928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:22.903928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:22.903928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:22.903928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:22.903928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:22.903928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:22.903928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:22.903928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:22.903928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:22.903928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:22.903928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:22.908514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:22.908514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:22.908514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:22.908514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:22.908514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:22.908514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:22.908514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:22.908514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:22.908514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:22.908514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:22.908514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:22.908514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:22.908514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:22.908514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:22.908514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:22.908514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:22.908514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:22.909861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:22.909861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:22.909861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:22.909861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:22.909861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 386/761] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:22.914973 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e5e278-a996-43a0-8f9f-6cc41f1d5150 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:22.941949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88011a65-850a-4443-b0a3-cfeb16f2aa22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:22.942788 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88011a65-850a-4443-b0a3-cfeb16f2aa22 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:22.943077 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88011a65-850a-4443-b0a3-cfeb16f2aa22 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:22.943359 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88011a65-850a-4443-b0a3-cfeb16f2aa22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:22.954713 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88011a65-850a-4443-b0a3-cfeb16f2aa22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:22.957052 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88011a65-850a-4443-b0a3-cfeb16f2aa22 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:22.977204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-88011a65-850a-4443-b0a3-cfeb16f2aa22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:22.977204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:22.977204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:22.977204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:22.977204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:22.977204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:22.977204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:22.977204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:22.977204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:22.977204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:22.977204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:22.977204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:22.977204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:22.977204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:22.977204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:22.982739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:22.982739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:22.982739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:22.982739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:22.982739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:22.982739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:22.982739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:22.982739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:22.982739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:22.982739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:22.982739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:22.982739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:22.982739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:22.982739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:22.982739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:22.982739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:22.982739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:22.983664 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:22.983664 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:22.983664 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:22.983664 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:22.983664 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 376/762] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:22.983664 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88011a65-850a-4443-b0a3-cfeb16f2aa22 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:23.036689 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e5e278-a996-43a0-8f9f-6cc41f1d5150 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:23.076465 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569052c-81f9-4822-964d-896b8bab0000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:23.077500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569052c-81f9-4822-964d-896b8bab0000 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:23.077972 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569052c-81f9-4822-964d-896b8bab0000 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:23.078458 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569052c-81f9-4822-964d-896b8bab0000 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:23.079468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569052c-81f9-4822-964d-896b8bab0000 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:23.080243 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569052c-81f9-4822-964d-896b8bab0000 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:23.081973 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0569052c-81f9-4822-964d-896b8bab0000 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:23.087938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 387/763] 10.222.0.16 () {56 vars in 1124 bytes} [Fri Jun 24 03:59:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:23.088813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88011a65-850a-4443-b0a3-cfeb16f2aa22 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:23.089252 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569052c-81f9-4822-964d-896b8bab0000 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:23.118123 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:23.128094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4f30e-ba40-43bf-834b-2a012744eb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:23.140370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4f30e-ba40-43bf-834b-2a012744eb7b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:23.140370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4f30e-ba40-43bf-834b-2a012744eb7b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:23.140370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4f30e-ba40-43bf-834b-2a012744eb7b None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:23.140370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4f30e-ba40-43bf-834b-2a012744eb7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:23.140370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4f30e-ba40-43bf-834b-2a012744eb7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:23.140370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4f30e-ba40-43bf-834b-2a012744eb7b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:23.143323 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:23.145408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 377/764] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:23.146360 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4f30e-ba40-43bf-834b-2a012744eb7b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:23.209339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0569052c-81f9-4822-964d-896b8bab0000 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:23.247176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ecc172-0026-4fdf-9c31-bd0d614187f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:23.248242 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecc172-0026-4fdf-9c31-bd0d614187f8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:23.248874 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecc172-0026-4fdf-9c31-bd0d614187f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:23.249376 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecc172-0026-4fdf-9c31-bd0d614187f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:23.250409 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ecc172-0026-4fdf-9c31-bd0d614187f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:23.250958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ecc172-0026-4fdf-9c31-bd0d614187f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:23.253285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ecc172-0026-4fdf-9c31-bd0d614187f8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:23.256059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4f30e-ba40-43bf-834b-2a012744eb7b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:23.266704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 388/765] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:23.268476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ecc172-0026-4fdf-9c31-bd0d614187f8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:23.292305 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f527dd0-4c28-46a4-8298-fcbfa039bad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:23.292901 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f527dd0-4c28-46a4-8298-fcbfa039bad5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:23.293188 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f527dd0-4c28-46a4-8298-fcbfa039bad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:23.293470 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f527dd0-4c28-46a4-8298-fcbfa039bad5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:23.294270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f527dd0-4c28-46a4-8298-fcbfa039bad5 None admin] RBAC: Adding query filter params (interface=public, service_id=7471bbf5fe8546779c3d0a70dcf88c68, region_id=RegionOne) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:23.294618 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f527dd0-4c28-46a4-8298-fcbfa039bad5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:23.295992 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f527dd0-4c28-46a4-8298-fcbfa039bad5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:23.303905 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 378/766] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Jun 24 03:59:23 2022] GET /identity/v3/endpoints?service_id=7471bbf5fe8546779c3d0a70dcf88c68&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:23.305124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa417c06-fd1a-405c-8481-4e1e0f740a40 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:23.305416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa417c06-fd1a-405c-8481-4e1e0f740a40 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:23.305754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa417c06-fd1a-405c-8481-4e1e0f740a40 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:23.310407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 379/767] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:23.310407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa417c06-fd1a-405c-8481-4e1e0f740a40 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:23.368573 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:23.384634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ecc172-0026-4fdf-9c31-bd0d614187f8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:23.392538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:23.420081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf46a6e-82f4-45fa-8e24-8542ed1cf29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:23.420866 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf46a6e-82f4-45fa-8e24-8542ed1cf29e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:23.421165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf46a6e-82f4-45fa-8e24-8542ed1cf29e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:23.421421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf46a6e-82f4-45fa-8e24-8542ed1cf29e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:23.427085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa417c06-fd1a-405c-8481-4e1e0f740a40 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:23.429230 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf46a6e-82f4-45fa-8e24-8542ed1cf29e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:23.430873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf46a6e-82f4-45fa-8e24-8542ed1cf29e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:23.445482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-1bf46a6e-82f4-45fa-8e24-8542ed1cf29e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:23.445482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:23.445482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:23.445482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:23.445482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:23.445482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:23.445482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:23.445482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:23.445482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:23.445482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:23.445482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:23.445482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:23.445482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:23.445482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:23.445482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:23.446680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:23.446680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:23.446680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:23.446680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:23.446680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:23.446680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:23.446680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:23.446680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:23.446680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:23.446680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:23.446680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:23.446680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:23.446680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:23.446680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:23.446680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:23.446680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:23.446680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:23.447798 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:23.447798 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:23.447798 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:23.447798 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:23.447798 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:23.447798 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:23.447798 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:23.447798 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:23.449794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 389/768] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:23.450869 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf46a6e-82f4-45fa-8e24-8542ed1cf29e None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:23.469097 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afcd7fb-28f2-4355-961c-df7beb496c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:23.469893 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcd7fb-28f2-4355-961c-df7beb496c75 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:23.470175 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcd7fb-28f2-4355-961c-df7beb496c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:23.471092 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afcd7fb-28f2-4355-961c-df7beb496c75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:23.478059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afcd7fb-28f2-4355-961c-df7beb496c75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:23.479542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afcd7fb-28f2-4355-961c-df7beb496c75 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:23.495109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-6afcd7fb-28f2-4355-961c-df7beb496c75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:23.495109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:23.495109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:23.495109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:23.495109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:23.495109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:23.495109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:23.495109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:23.495109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:23.495109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:23.495109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:23.495109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:23.495109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:23.495109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:23.495109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:23.496055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:23.496055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:23.496055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:23.496055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:23.496055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:23.496055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:23.496055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:23.496055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:23.496055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:23.496055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:23.496055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:23.496055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:23.496055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:23.496055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:23.496055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:23.496055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:23.496055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:23.496976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:23.496976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:23.496976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:23.496976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:23.496976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:23.496976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:23.496976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:23.496976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:23.496976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 380/769] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:23.499792 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c6d2e-5158-415d-893b-fb11bcc7196e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:23.499792 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c6d2e-5158-415d-893b-fb11bcc7196e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:23.499792 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78c6d2e-5158-415d-893b-fb11bcc7196e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:23.518794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d78c6d2e-5158-415d-893b-fb11bcc7196e None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:23.520576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d78c6d2e-5158-415d-893b-fb11bcc7196e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:23.576217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf46a6e-82f4-45fa-8e24-8542ed1cf29e None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:23.594532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 381/770] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:23.595928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c6d2e-5158-415d-893b-fb11bcc7196e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:23.613353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed217ab5-2c1b-41c0-8618-d8fb8fa40d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:23.614783 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed217ab5-2c1b-41c0-8618-d8fb8fa40d0c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:23.615292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed217ab5-2c1b-41c0-8618-d8fb8fa40d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:23.615809 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed217ab5-2c1b-41c0-8618-d8fb8fa40d0c None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:23.616703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed217ab5-2c1b-41c0-8618-d8fb8fa40d0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:23.618305 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed217ab5-2c1b-41c0-8618-d8fb8fa40d0c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:23.631049 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 390/771] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Jun 24 03:59:23 2022] GET /identity/v3/services => generated 772 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:23.632375 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed217ab5-2c1b-41c0-8618-d8fb8fa40d0c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:23.644124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:23.682022 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:23.714862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78c6d2e-5158-415d-893b-fb11bcc7196e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:23.754898 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f9291-0d10-4c1a-adcb-f12db3c307de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:23.755570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f9291-0d10-4c1a-adcb-f12db3c307de None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:23.755852 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f9291-0d10-4c1a-adcb-f12db3c307de None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:23.756127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f9291-0d10-4c1a-adcb-f12db3c307de None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:23.756925 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f9291-0d10-4c1a-adcb-f12db3c307de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:23.757237 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f9291-0d10-4c1a-adcb-f12db3c307de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:23.758741 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f9291-0d10-4c1a-adcb-f12db3c307de None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:23.776225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed217ab5-2c1b-41c0-8618-d8fb8fa40d0c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:23.778089 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 382/772] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:23.779093 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f9291-0d10-4c1a-adcb-f12db3c307de None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:23.818925 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e773722-0019-44a2-8e45-3d2cc0ee9df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:23.819595 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e773722-0019-44a2-8e45-3d2cc0ee9df3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:23.819882 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e773722-0019-44a2-8e45-3d2cc0ee9df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:23.820158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e773722-0019-44a2-8e45-3d2cc0ee9df3 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:23.820957 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e773722-0019-44a2-8e45-3d2cc0ee9df3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:23.821273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e773722-0019-44a2-8e45-3d2cc0ee9df3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:23.822765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e773722-0019-44a2-8e45-3d2cc0ee9df3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:23.839967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 391/773] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:23.841169 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e773722-0019-44a2-8e45-3d2cc0ee9df3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:23.901689 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:23.905095 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f9291-0d10-4c1a-adcb-f12db3c307de None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:23.952954 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e828f2-1a6b-4bf2-987f-a44c8412ecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:23.955520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e828f2-1a6b-4bf2-987f-a44c8412ecb4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:23.955994 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e828f2-1a6b-4bf2-987f-a44c8412ecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:23.956458 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e828f2-1a6b-4bf2-987f-a44c8412ecb4 None admin] PATH_INFO: `/v3/services/network` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:23.957296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e828f2-1a6b-4bf2-987f-a44c8412ecb4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:23.958936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e828f2-1a6b-4bf2-987f-a44c8412ecb4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:23.976223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-26e828f2-1a6b-4bf2-987f-a44c8412ecb4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 03:59:23.976223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:23.976223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:23.976223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:23.976223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:23.976223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:23.976223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:23.976223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:23.976223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 03:59:23.976223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:23.976223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:23.976223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:23.976223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:23.976223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:23.976223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:23.976223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:23.976223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:23.983606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:23.983606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:23.983606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:23.983606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:23.983606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:23.983606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:23.983606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:23.983606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:23.983606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:23.983606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:23.983606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:23.983606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:23.983606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:23.983606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:23.983606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:23.983606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:23.984480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:23.984480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:23.984480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:23.984480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:23.984480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:23.984480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:23.984480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:23.984480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:23.984480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:23.984480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:23.984480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:23.984480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:23.984480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 03:59:23.984480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:23.985231 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:23.985312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 383/774] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:23 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:23.985312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e828f2-1a6b-4bf2-987f-a44c8412ecb4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:23.998230 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e773722-0019-44a2-8e45-3d2cc0ee9df3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:24.049332 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1fbc0-7fa4-4b30-a9ad-aaaa87c6c881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:24.050833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1fbc0-7fa4-4b30-a9ad-aaaa87c6c881 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:24.051676 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1fbc0-7fa4-4b30-a9ad-aaaa87c6c881 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:24.052170 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a1fbc0-7fa4-4b30-a9ad-aaaa87c6c881 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:24.053237 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a1fbc0-7fa4-4b30-a9ad-aaaa87c6c881 None admin] RBAC: Adding query filter params (role.id=3073b5cac26f43089c9ed8c6a5102cff, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=98f58f91d17949198e66883d234a9e40) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:24.053730 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a1fbc0-7fa4-4b30-a9ad-aaaa87c6c881 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:24.055397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a1fbc0-7fa4-4b30-a9ad-aaaa87c6c881 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:24.070557 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 392/775] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:59:23 2022] GET /identity/v3/role_assignments?user.id=98f58f91d17949198e66883d234a9e40&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=3073b5cac26f43089c9ed8c6a5102cff => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:24.072052 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1fbc0-7fa4-4b30-a9ad-aaaa87c6c881 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:24.107066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e828f2-1a6b-4bf2-987f-a44c8412ecb4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:24.150057 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e55839-c16e-42f7-90f3-18d950999e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:24.150766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e55839-c16e-42f7-90f3-18d950999e85 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:24.151050 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e55839-c16e-42f7-90f3-18d950999e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:24.151325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e55839-c16e-42f7-90f3-18d950999e85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:24.152758 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e55839-c16e-42f7-90f3-18d950999e85 None admin] RBAC: Adding query filter params (role.id=f0f7e28cbe314921afdfb657b163436f, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=6d8e1f1e3c7e42cc9fa6f82aaac55129) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:24.153085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e55839-c16e-42f7-90f3-18d950999e85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:24.154742 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e55839-c16e-42f7-90f3-18d950999e85 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:24.163040 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 384/776] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:59:23 2022] GET /identity/v3/role_assignments?user.id=6d8e1f1e3c7e42cc9fa6f82aaac55129&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=f0f7e28cbe314921afdfb657b163436f => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:24.293284 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a1fbc0-7fa4-4b30-a9ad-aaaa87c6c881 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:24.344907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e89804d-049c-4eba-a577-69268f8ed5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:24.347257 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89804d-049c-4eba-a577-69268f8ed5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:24.347794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89804d-049c-4eba-a577-69268f8ed5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:24.348286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e89804d-049c-4eba-a577-69268f8ed5a0 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:24.349230 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89804d-049c-4eba-a577-69268f8ed5a0 None admin] RBAC: Adding query filter params (name=network) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:24.349743 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89804d-049c-4eba-a577-69268f8ed5a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:24.351371 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e89804d-049c-4eba-a577-69268f8ed5a0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:24.361802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 393/777] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jun 24 03:59:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:24.363464 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e55839-c16e-42f7-90f3-18d950999e85 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:24.395078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:24.453834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af490dd4-cc35-4b58-a842-a3ffaf12f5e4 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:24.454592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af490dd4-cc35-4b58-a842-a3ffaf12f5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:24.455640 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af490dd4-cc35-4b58-a842-a3ffaf12f5e4 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:24.458464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 394/778] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:24.467021 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a447fae-595f-45a2-bcf7-b89b8b27a449 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:24.467310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a447fae-595f-45a2-bcf7-b89b8b27a449 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:24.467591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a447fae-595f-45a2-bcf7-b89b8b27a449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:24.492181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7a447fae-595f-45a2-bcf7-b89b8b27a449 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:24.493390 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7a447fae-595f-45a2-bcf7-b89b8b27a449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:24.513414 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e55839-c16e-42f7-90f3-18d950999e85 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:24.526746 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:24.589206 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527ce0e-709f-4335-b930-bd06bfa64b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:24.589889 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527ce0e-709f-4335-b930-bd06bfa64b45 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:24.590178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527ce0e-709f-4335-b930-bd06bfa64b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:24.590498 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b527ce0e-709f-4335-b930-bd06bfa64b45 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:24.591224 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527ce0e-709f-4335-b930-bd06bfa64b45 None admin] RBAC: Adding query filter params (type=network) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:24.591539 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527ce0e-709f-4335-b930-bd06bfa64b45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:24.592929 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b527ce0e-709f-4335-b930-bd06bfa64b45 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:24.603561 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 385/779] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jun 24 03:59:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:24.609314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527ce0e-709f-4335-b930-bd06bfa64b45 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:24.633185 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 395/780] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:24.679357 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a447fae-595f-45a2-bcf7-b89b8b27a449 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:24.790837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:24.808100 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b527ce0e-709f-4335-b930-bd06bfa64b45 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:24.847887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac311740-50db-48e6-9d3e-17a7d52810b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:24.848575 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac311740-50db-48e6-9d3e-17a7d52810b8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:24.848864 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac311740-50db-48e6-9d3e-17a7d52810b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:24.849145 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac311740-50db-48e6-9d3e-17a7d52810b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:24.849960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac311740-50db-48e6-9d3e-17a7d52810b8 None admin] RBAC: Adding query filter params (interface=public, service_id=b028926e2b9c49edbf9d2d494b84c3ec, region_id=RegionOne) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:24.850279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac311740-50db-48e6-9d3e-17a7d52810b8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:24.851700 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac311740-50db-48e6-9d3e-17a7d52810b8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:24.856213 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a447fae-595f-45a2-bcf7-b89b8b27a449 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:24.867600 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 386/781] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Jun 24 03:59:24 2022] GET /identity/v3/endpoints?service_id=b028926e2b9c49edbf9d2d494b84c3ec&interface=public®ion_id=RegionOne => generated 194 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:24.878906 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac311740-50db-48e6-9d3e-17a7d52810b8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:24.903703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdf7ef-9748-4329-acb3-42ae0fc0dd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:24.904424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdf7ef-9748-4329-acb3-42ae0fc0dd4f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:24.904710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdf7ef-9748-4329-acb3-42ae0fc0dd4f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:24.904989 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdf7ef-9748-4329-acb3-42ae0fc0dd4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:24.931361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bdf7ef-9748-4329-acb3-42ae0fc0dd4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:24.939515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-59bdf7ef-9748-4329-acb3-42ae0fc0dd4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:59:24.939515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:24.939515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:24.939515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:24.939515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:24.939515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:24.939515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:24.939515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:24.939515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:24.939515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:24.939515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:24.939515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:24.939515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:24.939515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:24.939515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:24.939515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:24.940444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:24.940444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:24.940444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:24.940444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:24.940444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:24.940444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:24.940444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:24.940444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:24.940444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:24.940444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:24.940444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:24.940444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:24.940444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:24.940444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:24.940444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:24.940444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:24.941293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:24.941293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:24.941293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:24.941293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:24.941293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:24.941293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:24.941293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:24.941293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:24.941293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:24.941293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:24.941293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:59:24.941293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:24.942668 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 396/782] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:59:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:24.948114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdf7ef-9748-4329-acb3-42ae0fc0dd4f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:24.971225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:25.021607 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac311740-50db-48e6-9d3e-17a7d52810b8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:25.112508 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdf7ef-9748-4329-acb3-42ae0fc0dd4f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:25.113921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b1c9c-e8e9-42ed-8443-12e822644218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:25.114774 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b1c9c-e8e9-42ed-8443-12e822644218 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:25.115227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b1c9c-e8e9-42ed-8443-12e822644218 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:25.115644 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b1c9c-e8e9-42ed-8443-12e822644218 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:25.116464 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b1c9c-e8e9-42ed-8443-12e822644218 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:25.117972 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b1c9c-e8e9-42ed-8443-12e822644218 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:25.129561 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 387/783] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Jun 24 03:59:24 2022] GET /identity/v3/services => generated 772 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:25.156991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16648e19-a0cf-4d19-8935-a71f005a6f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:25.158528 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16648e19-a0cf-4d19-8935-a71f005a6f53 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:25.158961 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16648e19-a0cf-4d19-8935-a71f005a6f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:25.159443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16648e19-a0cf-4d19-8935-a71f005a6f53 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:25.160353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16648e19-a0cf-4d19-8935-a71f005a6f53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:25.160836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16648e19-a0cf-4d19-8935-a71f005a6f53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:25.162419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16648e19-a0cf-4d19-8935-a71f005a6f53 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:25.173024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 397/784] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:59:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:25.178538 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b1c9c-e8e9-42ed-8443-12e822644218 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:25.228199 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:25.322428 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b1c9c-e8e9-42ed-8443-12e822644218 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:25.357768 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1a198-ae71-4e6a-8d22-d0e037fa5eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:25.358594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1a198-ae71-4e6a-8d22-d0e037fa5eca None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:25.358880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1a198-ae71-4e6a-8d22-d0e037fa5eca None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:25.359159 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1a198-ae71-4e6a-8d22-d0e037fa5eca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:25.372112 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1a198-ae71-4e6a-8d22-d0e037fa5eca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:25.373577 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1a198-ae71-4e6a-8d22-d0e037fa5eca None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:25.392532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-9bd1a198-ae71-4e6a-8d22-d0e037fa5eca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:25.392532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:25.392532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:25.392532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:25.392532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:25.392532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:25.392532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:25.392532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:25.392532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:25.392532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:25.392532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:25.392532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:25.392532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:25.392532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:25.392532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:25.393899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:25.393899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:25.393899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:25.393899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:25.393899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:25.393899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:25.393899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:25.393899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:25.393899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:25.393899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:25.393899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:25.393899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:25.393899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:25.393899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:25.393899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:25.393899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:25.393899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:25.395371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:25.395371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:25.395371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:25.395371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:25.395371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 388/785] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:25.399450 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16648e19-a0cf-4d19-8935-a71f005a6f53 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:25.485868 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:25.514994 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16648e19-a0cf-4d19-8935-a71f005a6f53 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:25.583672 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556875b7-e2da-4c79-9f15-1c9d1df73a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:25.584416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556875b7-e2da-4c79-9f15-1c9d1df73a54 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:25.584704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556875b7-e2da-4c79-9f15-1c9d1df73a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:25.584978 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556875b7-e2da-4c79-9f15-1c9d1df73a54 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:25.585830 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556875b7-e2da-4c79-9f15-1c9d1df73a54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:25.586142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556875b7-e2da-4c79-9f15-1c9d1df73a54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:25.587556 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556875b7-e2da-4c79-9f15-1c9d1df73a54 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:25.610816 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 398/786] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:25.616491 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1a198-ae71-4e6a-8d22-d0e037fa5eca None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:25.731627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:25.775941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1a198-ae71-4e6a-8d22-d0e037fa5eca None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:25.810140 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4bc3e7-2011-4321-a5bd-beea7105e738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:25.810970 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bc3e7-2011-4321-a5bd-beea7105e738 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:25.811255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bc3e7-2011-4321-a5bd-beea7105e738 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:25.811534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bc3e7-2011-4321-a5bd-beea7105e738 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:25.819978 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4bc3e7-2011-4321-a5bd-beea7105e738 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:25.821436 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4bc3e7-2011-4321-a5bd-beea7105e738 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:25.836285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-cb4bc3e7-2011-4321-a5bd-beea7105e738 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 03:59:25.836285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:25.836285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:25.836285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:25.836285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:25.836285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:25.836285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:25.836285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:25.836285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:25.836285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:25.836285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:25.836285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:25.836285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:25.836285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:25.836285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:25.836285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:25.837229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:25.837229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:25.837229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:25.837229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:25.837229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:25.837229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:25.837229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:25.837229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:25.837229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:25.837229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:25.837229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:25.837229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:25.837229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:25.837229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:25.837229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:25.837229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:25.837229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:25.837229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:25.838183 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:25.838183 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:25.838183 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:25.838183 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:25.838183 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:25.838183 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:25.838183 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:25.838183 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:25.838183 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 03:59:25.838183 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:25.838183 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 389/787] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Jun 24 03:59:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:25.844848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556875b7-e2da-4c79-9f15-1c9d1df73a54 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:25.933594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd991473-16ca-4a83-afda-eed9324b5790 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:25.933930 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd991473-16ca-4a83-afda-eed9324b5790 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:25.934162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd991473-16ca-4a83-afda-eed9324b5790 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:25.935860 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 390/788] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:25.953828 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e3d10-6fc7-4805-9f52-c94dd9338318 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:25.954138 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e3d10-6fc7-4805-9f52-c94dd9338318 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:25.954428 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e3d10-6fc7-4805-9f52-c94dd9338318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:25.986123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556875b7-e2da-4c79-9f15-1c9d1df73a54 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:25.987402 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b06e3d10-6fc7-4805-9f52-c94dd9338318 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:25.988607 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b06e3d10-6fc7-4805-9f52-c94dd9338318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:26.051924 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:26.062305 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36484f-02c1-482d-a43f-5fed7f81d8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:26.063047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36484f-02c1-482d-a43f-5fed7f81d8be None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:26.063327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36484f-02c1-482d-a43f-5fed7f81d8be None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:26.063599 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36484f-02c1-482d-a43f-5fed7f81d8be None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:26.065702 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36484f-02c1-482d-a43f-5fed7f81d8be None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:26.066014 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36484f-02c1-482d-a43f-5fed7f81d8be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:26.068604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36484f-02c1-482d-a43f-5fed7f81d8be None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:26.106532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 399/789] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Jun 24 03:59:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:26.109956 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36484f-02c1-482d-a43f-5fed7f81d8be None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:26.129152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 391/790] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 03:59:26.163060 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e3d10-6fc7-4805-9f52-c94dd9338318 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:26.277074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:26.286904 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36484f-02c1-482d-a43f-5fed7f81d8be None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:26.354052 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e3d10-6fc7-4805-9f52-c94dd9338318 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:26.368463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec638a-d01a-4ec1-946e-88742ab05137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:26.369395 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec638a-d01a-4ec1-946e-88742ab05137 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:26.369683 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec638a-d01a-4ec1-946e-88742ab05137 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:26.369974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec638a-d01a-4ec1-946e-88742ab05137 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:26.391821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee408e-583b-40b9-89b1-6976f9a5d0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:26.392675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee408e-583b-40b9-89b1-6976f9a5d0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:26.392959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee408e-583b-40b9-89b1-6976f9a5d0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:26.393239 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee408e-583b-40b9-89b1-6976f9a5d0a4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:26.393898 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee408e-583b-40b9-89b1-6976f9a5d0a4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:26.395302 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee408e-583b-40b9-89b1-6976f9a5d0a4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:26.409304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-61ee408e-583b-40b9-89b1-6976f9a5d0a4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 03:59:26.409304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:26.409304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:26.409304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:26.409304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:26.409304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:26.409304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:26.409304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:26.409304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 03:59:26.409304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:26.409304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:26.409304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:26.409304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:26.409304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:26.409304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:26.409304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:26.410267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:26.410267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:26.410267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:26.410267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:26.410267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:26.410267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:26.410267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:26.410267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:26.410267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:26.410267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:26.410267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:26.410267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:26.410267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:26.410267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:26.410267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:26.410267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:26.411178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:26.411178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:26.411178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:26.411178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:26.411178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:26.411178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:26.411178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:26.411178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:26.411178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:26.411178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:26.411178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:26.411178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:26.411178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:26.411178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 03:59:26.411178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:26.411178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 392/791] 10.222.0.16 () {56 vars in 1129 bytes} [Fri Jun 24 03:59:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:26.420233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee408e-583b-40b9-89b1-6976f9a5d0a4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:26.479672 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:26.504761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec638a-d01a-4ec1-946e-88742ab05137 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:26.506249 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec638a-d01a-4ec1-946e-88742ab05137 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:26.534875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee408e-583b-40b9-89b1-6976f9a5d0a4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:26.561602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:26.571415 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda7f5b-9f30-4d6f-b1e8-73414db6c7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:26.572312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda7f5b-9f30-4d6f-b1e8-73414db6c7e9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:26.572660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda7f5b-9f30-4d6f-b1e8-73414db6c7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:26.573002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda7f5b-9f30-4d6f-b1e8-73414db6c7e9 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:26.573891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda7f5b-9f30-4d6f-b1e8-73414db6c7e9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:26.574262 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda7f5b-9f30-4d6f-b1e8-73414db6c7e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:26.575738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda7f5b-9f30-4d6f-b1e8-73414db6c7e9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:26.582060 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 393/792] 10.222.0.16 () {56 vars in 1124 bytes} [Fri Jun 24 03:59:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:26.588040 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda7f5b-9f30-4d6f-b1e8-73414db6c7e9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:26.677853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 400/793] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:59:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 03:59:26.679604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ea1a2-f0b8-4044-a84c-0729d68864ad None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:26.679963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ea1a2-f0b8-4044-a84c-0729d68864ad None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:26.680245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ea1a2-f0b8-4044-a84c-0729d68864ad None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:26.685553 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 401/794] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:26.686454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ea1a2-f0b8-4044-a84c-0729d68864ad None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:26.707171 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda7f5b-9f30-4d6f-b1e8-73414db6c7e9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:26.743854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872eaac9-e433-49a3-88f7-7b540850b142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:26.744774 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872eaac9-e433-49a3-88f7-7b540850b142 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:26.745062 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872eaac9-e433-49a3-88f7-7b540850b142 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:26.745344 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872eaac9-e433-49a3-88f7-7b540850b142 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:26.746055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872eaac9-e433-49a3-88f7-7b540850b142 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:26.746389 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872eaac9-e433-49a3-88f7-7b540850b142 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:26.747790 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872eaac9-e433-49a3-88f7-7b540850b142 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:26.754077 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 394/795] 10.222.0.16 () {56 vars in 1124 bytes} [Fri Jun 24 03:59:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:26.755448 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f70ecd-e2e3-4e22-8db2-bd433035d6ac None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:26.755735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f70ecd-e2e3-4e22-8db2-bd433035d6ac None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:26.756013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f70ecd-e2e3-4e22-8db2-bd433035d6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:26.774723 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f2f70ecd-e2e3-4e22-8db2-bd433035d6ac None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:26.776080 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f2f70ecd-e2e3-4e22-8db2-bd433035d6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:26.787414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:26.802004 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ea1a2-f0b8-4044-a84c-0729d68864ad None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:26.834458 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:26.838946 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56d65f-ebe7-4399-9388-245797323dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:26.839402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56d65f-ebe7-4399-9388-245797323dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:26.839694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56d65f-ebe7-4399-9388-245797323dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:26.839972 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56d65f-ebe7-4399-9388-245797323dd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:26.850770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56d65f-ebe7-4399-9388-245797323dd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:26.852262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c56d65f-ebe7-4399-9388-245797323dd2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:26.857979 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 395/796] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:26.857979 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7834c-a316-44db-b252-f8c4df857cb9 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:26.857979 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7834c-a316-44db-b252-f8c4df857cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:26.857979 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7834c-a316-44db-b252-f8c4df857cb9 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:26.857979 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 396/797] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:26.857979 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7834c-a316-44db-b252-f8c4df857cb9 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:26.884949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-2c56d65f-ebe7-4399-9388-245797323dd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:26.884949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:26.884949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:26.884949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:26.884949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:26.884949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:26.884949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:26.884949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:26.884949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:26.884949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:26.884949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:26.884949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:26.884949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:26.884949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:26.884949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:26.886085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:26.886085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:26.886085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:26.886085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:26.886085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:26.886085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:26.886085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:26.886085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:26.886085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:26.886085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:26.886085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:26.886085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:26.886085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:26.886085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:26.886085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:26.886085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:26.886085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:26.887149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:26.887149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:26.887149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:26.887149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:26.887149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 402/798] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:26.887995 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3979e-2a9f-46b5-9c30-e5e3c6232350 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:26.887995 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3979e-2a9f-46b5-9c30-e5e3c6232350 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:26.888138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3979e-2a9f-46b5-9c30-e5e3c6232350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:26.914326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ecb3979e-2a9f-46b5-9c30-e5e3c6232350 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:26.915913 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ecb3979e-2a9f-46b5-9c30-e5e3c6232350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:26.995759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 403/799] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:26.996582 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7834c-a316-44db-b252-f8c4df857cb9 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:27.000249 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3979e-2a9f-46b5-9c30-e5e3c6232350 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:27.034400 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecbffb9-5d44-469c-a862-90209d2798bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:27.035073 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecbffb9-5d44-469c-a862-90209d2798bf None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:27.035358 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecbffb9-5d44-469c-a862-90209d2798bf None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:27.035636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecbffb9-5d44-469c-a862-90209d2798bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:27.036378 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecbffb9-5d44-469c-a862-90209d2798bf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:27.038017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecbffb9-5d44-469c-a862-90209d2798bf None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:27.088778 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:27.118898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3979e-2a9f-46b5-9c30-e5e3c6232350 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:27.153006 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 397/800] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Jun 24 03:59:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:59:27.159193 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8e315-deac-44ba-8f82-c1839a96fc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:27.160304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8e315-deac-44ba-8f82-c1839a96fc04 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:27.160823 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8e315-deac-44ba-8f82-c1839a96fc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:27.161260 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e8e315-deac-44ba-8f82-c1839a96fc04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:27.163527 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecbffb9-5d44-469c-a862-90209d2798bf None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:27.174863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e8e315-deac-44ba-8f82-c1839a96fc04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:27.183811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-b1e8e315-deac-44ba-8f82-c1839a96fc04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:27.183811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:27.183811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:27.183811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:27.183811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:27.183811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:27.183811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:27.183811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:27.183811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:27.183811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:27.183811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:27.183811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:27.183811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:27.183811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:27.183811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:27.183811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:27.185106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:27.185106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:27.185106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:27.185106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:27.185106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:27.185106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:27.185106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:27.185106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:27.185106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:27.185106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:27.185106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:27.185106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:27.185106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:27.185106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:27.185106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:27.185106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:27.186225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:27.186225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:27.186225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:27.186225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:27.186225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:27.186225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:27.186225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:27.186225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:27.186225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:27.186225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:27.186225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:27.186225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:27.188848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 404/801] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:59:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:27.190708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8e315-deac-44ba-8f82-c1839a96fc04 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:27.237527 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:27.326261 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecbffb9-5d44-469c-a862-90209d2798bf None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:27.327296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e8e315-deac-44ba-8f82-c1839a96fc04 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:27.366259 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:27.368823 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11a297-ab2e-4a76-8767-e01fc70aa7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:27.369756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11a297-ab2e-4a76-8767-e01fc70aa7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:27.370040 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11a297-ab2e-4a76-8767-e01fc70aa7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:27.370313 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc11a297-ab2e-4a76-8767-e01fc70aa7e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:27.371135 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11a297-ab2e-4a76-8767-e01fc70aa7e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:27.371440 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11a297-ab2e-4a76-8767-e01fc70aa7e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:27.372810 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc11a297-ab2e-4a76-8767-e01fc70aa7e4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:27.376288 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a909c5-d6d7-42e8-b07d-034f7a85f28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:27.377834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a909c5-d6d7-42e8-b07d-034f7a85f28d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:27.378206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a909c5-d6d7-42e8-b07d-034f7a85f28d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:27.379396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a909c5-d6d7-42e8-b07d-034f7a85f28d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:27.388101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 398/802] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:27.388101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11a297-ab2e-4a76-8767-e01fc70aa7e4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:27.400514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a909c5-d6d7-42e8-b07d-034f7a85f28d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:27.408966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-d6a909c5-d6d7-42e8-b07d-034f7a85f28d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:59:27.408966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:27.408966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:27.408966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:27.408966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:27.408966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:27.408966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:27.408966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:27.408966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:27.408966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:27.408966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:27.408966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:27.408966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:27.408966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:27.408966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:27.408966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:27.409953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:27.409953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:27.409953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:27.409953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:27.409953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:27.409953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:27.409953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:27.409953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:27.409953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:27.409953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:27.409953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:27.409953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:27.409953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:27.409953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:27.409953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:27.409953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:27.410846 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:27.410846 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:27.410846 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:27.410846 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:27.410846 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:27.410846 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:27.410846 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:27.410846 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:27.410846 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:27.410846 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:27.410846 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:59:27.410846 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:27.410846 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 405/803] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:59:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:27.411553 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a909c5-d6d7-42e8-b07d-034f7a85f28d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:27.510447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:27.514041 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc11a297-ab2e-4a76-8767-e01fc70aa7e4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:27.541231 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a909c5-d6d7-42e8-b07d-034f7a85f28d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:27.549416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e270e-376a-40b9-a109-844df9ea6851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:27.550086 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e270e-376a-40b9-a109-844df9ea6851 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:27.550387 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e270e-376a-40b9-a109-844df9ea6851 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:27.550688 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e270e-376a-40b9-a109-844df9ea6851 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:27.551438 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e270e-376a-40b9-a109-844df9ea6851 None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:27.551739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e270e-376a-40b9-a109-844df9ea6851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:27.553114 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e270e-376a-40b9-a109-844df9ea6851 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:27.566408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 399/804] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 03:59:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:27.566408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e270e-376a-40b9-a109-844df9ea6851 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:27.591252 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ba29c-39b6-4168-bcbd-68fb3f1cd9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:27.592110 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ba29c-39b6-4168-bcbd-68fb3f1cd9de None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:27.592525 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ba29c-39b6-4168-bcbd-68fb3f1cd9de None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:27.592881 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ba29c-39b6-4168-bcbd-68fb3f1cd9de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:27.600810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ba29c-39b6-4168-bcbd-68fb3f1cd9de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:27.602596 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ba29c-39b6-4168-bcbd-68fb3f1cd9de None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:27.616837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-6d9ba29c-39b6-4168-bcbd-68fb3f1cd9de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:27.616837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:27.616837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:27.616837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:27.616837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:27.616837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:27.616837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:27.616837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:27.616837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:27.616837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:27.616837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:27.616837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:27.616837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:27.616837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:27.616837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:27.617971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:27.617971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:27.617971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:27.617971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:27.617971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:27.617971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:27.617971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:27.617971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:27.617971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:27.617971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:27.617971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:27.617971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:27.617971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:27.617971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:27.617971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:27.617971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:27.617971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:27.619344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:27.619344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:27.619344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:27.619344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:27.619344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:27.619344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:27.619344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:27.619344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:27.621098 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 406/805] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:27.621982 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ba29c-39b6-4168-bcbd-68fb3f1cd9de None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:27.672769 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:27.686786 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e270e-376a-40b9-a109-844df9ea6851 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:27.727934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7507a-60d8-4c21-9aef-69e30001e60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:27.729487 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7507a-60d8-4c21-9aef-69e30001e60d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:27.729907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ba29c-39b6-4168-bcbd-68fb3f1cd9de None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:27.730132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7507a-60d8-4c21-9aef-69e30001e60d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:27.730594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7507a-60d8-4c21-9aef-69e30001e60d None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:27.731492 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7507a-60d8-4c21-9aef-69e30001e60d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:27.732093 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7507a-60d8-4c21-9aef-69e30001e60d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:27.733716 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7507a-60d8-4c21-9aef-69e30001e60d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:27.743700 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 400/806] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:59:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:27.745095 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7507a-60d8-4c21-9aef-69e30001e60d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:27.771717 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d19a8-6e25-4cd8-bc73-cef3c55b75ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:27.772276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:27.772872 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d19a8-6e25-4cd8-bc73-cef3c55b75ff None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:27.773139 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d19a8-6e25-4cd8-bc73-cef3c55b75ff None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:27.773540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d19a8-6e25-4cd8-bc73-cef3c55b75ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:27.785530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2d19a8-6e25-4cd8-bc73-cef3c55b75ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:27.787696 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2d19a8-6e25-4cd8-bc73-cef3c55b75ff None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:27.804840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-cf2d19a8-6e25-4cd8-bc73-cef3c55b75ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:27.804840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:27.804840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:27.804840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:27.804840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:27.804840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:27.804840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:27.804840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:27.804840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:27.804840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:27.804840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:27.804840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:27.804840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:27.804840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:27.804840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:27.805825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:27.805825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:27.805825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:27.805825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:27.805825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:27.805825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:27.805825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:27.805825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:27.805825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:27.805825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:27.805825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:27.805825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:27.805825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:27.805825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:27.805825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:27.805825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:27.805825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:27.807135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:27.807135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:27.807135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:27.807135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:27.807135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 407/807] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:27.807135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d19a8-6e25-4cd8-bc73-cef3c55b75ff None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:27.857370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7507a-60d8-4c21-9aef-69e30001e60d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:27.893321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13aa96c-db7b-4098-a1da-310c72819ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:27.893987 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13aa96c-db7b-4098-a1da-310c72819ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:27.894275 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13aa96c-db7b-4098-a1da-310c72819ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:27.894585 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13aa96c-db7b-4098-a1da-310c72819ea6 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:27.895378 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13aa96c-db7b-4098-a1da-310c72819ea6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:27.895691 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13aa96c-db7b-4098-a1da-310c72819ea6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:27.897136 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13aa96c-db7b-4098-a1da-310c72819ea6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:27.909925 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 401/808] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:27.911522 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea1889-673b-484b-934c-0fe9fbbab9e4 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:27.912041 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea1889-673b-484b-934c-0fe9fbbab9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:27.912585 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea1889-673b-484b-934c-0fe9fbbab9e4 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:27.914280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 402/809] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:27.915227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea1889-673b-484b-934c-0fe9fbbab9e4 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:27.926810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:27.932303 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d19a8-6e25-4cd8-bc73-cef3c55b75ff None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:27.968872 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c1d92-735a-437c-8d18-477fd41623c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:27.969673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c1d92-735a-437c-8d18-477fd41623c8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:27.969958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c1d92-735a-437c-8d18-477fd41623c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:27.970287 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773c1d92-735a-437c-8d18-477fd41623c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:27.981050 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c1d92-735a-437c-8d18-477fd41623c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:27.982536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773c1d92-735a-437c-8d18-477fd41623c8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:28.001941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-773c1d92-735a-437c-8d18-477fd41623c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:28.001941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:28.001941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:28.001941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:28.001941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:28.001941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:28.001941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:28.001941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:28.001941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:28.001941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:28.001941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:28.001941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:28.001941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:28.001941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:28.001941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:28.009073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:28.009073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:28.009073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:28.009073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:28.009073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:28.009073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:28.009073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:28.009073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:28.009073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:28.009073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:28.009073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:28.009073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:28.009073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:28.009073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:28.009073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:28.009073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:28.009073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:28.010173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:28.010173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:28.010173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:28.010173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:28.010173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 408/810] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:28.010173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c1d92-735a-437c-8d18-477fd41623c8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:28.018759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:28.041579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea1889-673b-484b-934c-0fe9fbbab9e4 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:28.081338 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f5b94-1eae-437e-84a2-860e5be5fccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:28.082192 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f5b94-1eae-437e-84a2-860e5be5fccc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:28.082507 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f5b94-1eae-437e-84a2-860e5be5fccc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:28.082790 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f5b94-1eae-437e-84a2-860e5be5fccc None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:28.083562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f5b94-1eae-437e-84a2-860e5be5fccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:28.083870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f5b94-1eae-437e-84a2-860e5be5fccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:28.085402 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056f5b94-1eae-437e-84a2-860e5be5fccc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:28.101777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 403/811] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:28.101777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c23b8-f174-4bbc-ae93-3f43e4a97748 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:28.101777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c23b8-f174-4bbc-ae93-3f43e4a97748 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:28.101777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c23b8-f174-4bbc-ae93-3f43e4a97748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:28.116272 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3e3c23b8-f174-4bbc-ae93-3f43e4a97748 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:28.117530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3e3c23b8-f174-4bbc-ae93-3f43e4a97748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:28.129754 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773c1d92-735a-437c-8d18-477fd41623c8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:28.165602 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb04dff-aea6-4ff3-be25-99c2330e9893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:28.166254 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb04dff-aea6-4ff3-be25-99c2330e9893 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:28.166571 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb04dff-aea6-4ff3-be25-99c2330e9893 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:28.166850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb04dff-aea6-4ff3-be25-99c2330e9893 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:28.167703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb04dff-aea6-4ff3-be25-99c2330e9893 None admin] RBAC: Adding query filter params (role.id=f0f7e28cbe314921afdfb657b163436f, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=80b7b3ec60db4de19bbf7d08f111f470) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:28.167988 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb04dff-aea6-4ff3-be25-99c2330e9893 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:28.169418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb04dff-aea6-4ff3-be25-99c2330e9893 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:28.175490 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 409/812] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:59:28 2022] GET /identity/v3/role_assignments?user.id=80b7b3ec60db4de19bbf7d08f111f470&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=f0f7e28cbe314921afdfb657b163436f => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:28.176262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb04dff-aea6-4ff3-be25-99c2330e9893 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:28.190752 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 404/813] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:28.191973 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c23b8-f174-4bbc-ae93-3f43e4a97748 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:28.214891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:28.294323 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:28.324608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c23b8-f174-4bbc-ae93-3f43e4a97748 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:28.335465 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb04dff-aea6-4ff3-be25-99c2330e9893 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:28.368725 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdee13f-2a25-4ffc-8dc7-9a342d1dea3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:28.368725 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdee13f-2a25-4ffc-8dc7-9a342d1dea3f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:28.368725 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdee13f-2a25-4ffc-8dc7-9a342d1dea3f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:28.368725 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdee13f-2a25-4ffc-8dc7-9a342d1dea3f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:28.379612 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdee13f-2a25-4ffc-8dc7-9a342d1dea3f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:28.380386 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdee13f-2a25-4ffc-8dc7-9a342d1dea3f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:28.383861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351828f-73e2-4281-b41f-75bcba62d5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:28.384554 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351828f-73e2-4281-b41f-75bcba62d5c0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:28.384876 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351828f-73e2-4281-b41f-75bcba62d5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:28.385159 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351828f-73e2-4281-b41f-75bcba62d5c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:28.385905 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351828f-73e2-4281-b41f-75bcba62d5c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:28.386215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351828f-73e2-4281-b41f-75bcba62d5c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:28.387916 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351828f-73e2-4281-b41f-75bcba62d5c0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:28.398165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-1bdee13f-2a25-4ffc-8dc7-9a342d1dea3f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 03:59:28.398165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:28.398165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:28.398165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:28.398165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:28.398165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:28.398165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:28.398165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:28.398165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:28.398165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:28.398165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:28.398165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:28.398165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:28.398165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:28.398165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:28.398165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:28.399322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:28.399322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:28.399322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:28.399322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:28.399322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:28.399322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:28.399322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:28.399322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:28.399322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:28.399322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:28.399322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:28.399322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:28.399322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:28.399322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:28.399322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:28.399322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:28.399322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:28.399322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:28.400293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:28.400293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:28.400293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:28.400293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:28.400293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:28.400293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:28.400293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:28.400293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:28.400293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 03:59:28.400293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:28.404889 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 405/815] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jun 24 03:59:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:28.405154 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 410/815] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:28.405800 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdee13f-2a25-4ffc-8dc7-9a342d1dea3f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:28.406033 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351828f-73e2-4281-b41f-75bcba62d5c0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:28.499391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:28.528127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351828f-73e2-4281-b41f-75bcba62d5c0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:28.540343 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdee13f-2a25-4ffc-8dc7-9a342d1dea3f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:28.569739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57654656-b98f-441b-ae77-67404fa4b8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:28.570903 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57654656-b98f-441b-ae77-67404fa4b8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:28.571322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57654656-b98f-441b-ae77-67404fa4b8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:28.571734 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57654656-b98f-441b-ae77-67404fa4b8d3 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:28.572835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57654656-b98f-441b-ae77-67404fa4b8d3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:28.573155 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57654656-b98f-441b-ae77-67404fa4b8d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:28.575340 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57654656-b98f-441b-ae77-67404fa4b8d3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:28.582867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c1465-1197-4f4f-978f-c2225eb1600d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:28.583769 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c1465-1197-4f4f-978f-c2225eb1600d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:28.584130 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c1465-1197-4f4f-978f-c2225eb1600d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:28.584487 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c1465-1197-4f4f-978f-c2225eb1600d None admin] PATH_INFO: `/v3/services/network` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:28.585221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c1465-1197-4f4f-978f-c2225eb1600d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:28.586927 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c1465-1197-4f4f-978f-c2225eb1600d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:28.590039 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:28.597788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 411/816] 10.222.0.16 () {56 vars in 1149 bytes} [Fri Jun 24 03:59:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:28.599603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57654656-b98f-441b-ae77-67404fa4b8d3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:28.608721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-712c1465-1197-4f4f-978f-c2225eb1600d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 03:59:28.608721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:28.608721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:28.608721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:28.608721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:28.608721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:28.608721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:28.608721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:28.608721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 03:59:28.608721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:28.608721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:28.608721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:28.608721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:28.608721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:28.608721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:28.608721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:28.608721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:28.610023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:28.610023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:28.610023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:28.610023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:28.610023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:28.610023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:28.610023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:28.610023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:28.610023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:28.610023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:28.610023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:28.610023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:28.610023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:28.610023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:28.610023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:28.610023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:28.611287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:28.611287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:28.611287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:28.611287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:28.611287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:28.611287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:28.611287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:28.611287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:28.611287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:28.611287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:28.611287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:28.611287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:28.611287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 03:59:28.611287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:28.612796 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 406/817] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:28 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:28.613674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c1465-1197-4f4f-978f-c2225eb1600d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:28.721996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c1465-1197-4f4f-978f-c2225eb1600d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:28.743349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57654656-b98f-441b-ae77-67404fa4b8d3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:28.763302 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b837d-9f39-4996-9b76-23bf3e53f767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:28.764370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b837d-9f39-4996-9b76-23bf3e53f767 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:28.764655 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b837d-9f39-4996-9b76-23bf3e53f767 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:28.764935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b837d-9f39-4996-9b76-23bf3e53f767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:28.780630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:28.781088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b837d-9f39-4996-9b76-23bf3e53f767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:28.783165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b837d-9f39-4996-9b76-23bf3e53f767 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:28.789098 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822aea4-d0b9-44fe-98f9-022216c671cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:28.794022 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822aea4-d0b9-44fe-98f9-022216c671cd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:28.794022 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822aea4-d0b9-44fe-98f9-022216c671cd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:28.794022 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e822aea4-d0b9-44fe-98f9-022216c671cd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:28.799942 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822aea4-d0b9-44fe-98f9-022216c671cd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:28.801516 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e822aea4-d0b9-44fe-98f9-022216c671cd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:28.802736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-cc5b837d-9f39-4996-9b76-23bf3e53f767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:28.802736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:28.802736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:28.802736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:28.802736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:28.802736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:28.802736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:28.802736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:28.802736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:28.802736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:28.802736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:28.802736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:28.802736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:28.802736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:28.802736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:28.803703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:28.803703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:28.803703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:28.803703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:28.803703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:28.803703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:28.803703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:28.803703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:28.803703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:28.803703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:28.803703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:28.803703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:28.803703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:28.803703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:28.803703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:28.803703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:28.803703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:28.804735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:28.804735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:28.804735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:28.804735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:28.809314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 407/818] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:28.810289 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b837d-9f39-4996-9b76-23bf3e53f767 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:28.818633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-e822aea4-d0b9-44fe-98f9-022216c671cd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 03:59:28.818633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:28.818633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:28.818633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:28.818633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:28.818633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:28.818633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:28.818633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:28.818633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:28.818633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:28.818633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:28.818633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:28.818633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:28.818633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:28.818633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:28.818633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:28.819806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:28.819806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:28.819806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:28.819806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:28.819806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:28.819806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:28.819806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:28.819806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:28.819806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:28.819806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:28.819806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:28.819806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:28.819806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:28.819806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:28.819806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:28.819806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:28.819806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:28.819806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:28.821889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:28.821889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:28.821889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:28.821889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:28.821889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:28.821889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:28.821889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:28.821889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:28.821889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 03:59:28.821889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:28.821889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 412/819] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:59:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:28.823023 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822aea4-d0b9-44fe-98f9-022216c671cd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:28.942312 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e822aea4-d0b9-44fe-98f9-022216c671cd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:28.985799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:28.991702 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c21cc-d0aa-49e8-8a78-14d670ac7375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:28.993045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c21cc-d0aa-49e8-8a78-14d670ac7375 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:28.993523 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c21cc-d0aa-49e8-8a78-14d670ac7375 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:28.993907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c21cc-d0aa-49e8-8a78-14d670ac7375 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:28.995356 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c21cc-d0aa-49e8-8a78-14d670ac7375 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:28.995773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c21cc-d0aa-49e8-8a78-14d670ac7375 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:28.997912 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994c21cc-d0aa-49e8-8a78-14d670ac7375 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:29.020101 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 413/820] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:29.020101 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c21cc-d0aa-49e8-8a78-14d670ac7375 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:29.059394 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b837d-9f39-4996-9b76-23bf3e53f767 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:29.089646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:29.096299 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2f425-0cc7-4957-a8c9-e74829f395e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:29.097234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2f425-0cc7-4957-a8c9-e74829f395e0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:29.097681 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2f425-0cc7-4957-a8c9-e74829f395e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:29.098110 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2f425-0cc7-4957-a8c9-e74829f395e0 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:29.098997 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2f425-0cc7-4957-a8c9-e74829f395e0 None admin] RBAC: Adding query filter params (name=network) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:29.101170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2f425-0cc7-4957-a8c9-e74829f395e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:29.101170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2f425-0cc7-4957-a8c9-e74829f395e0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:29.106224 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 408/821] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jun 24 03:59:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:29.107289 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2f425-0cc7-4957-a8c9-e74829f395e0 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:29.144870 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c21cc-d0aa-49e8-8a78-14d670ac7375 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:29.182535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed868ea6-1ea3-4baf-9dc0-37efdd8a3beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:29.183323 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed868ea6-1ea3-4baf-9dc0-37efdd8a3beb None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:29.183675 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed868ea6-1ea3-4baf-9dc0-37efdd8a3beb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:29.184053 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed868ea6-1ea3-4baf-9dc0-37efdd8a3beb None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:29.184966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed868ea6-1ea3-4baf-9dc0-37efdd8a3beb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:29.185350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed868ea6-1ea3-4baf-9dc0-37efdd8a3beb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:29.186980 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed868ea6-1ea3-4baf-9dc0-37efdd8a3beb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:29.206674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 414/822] 10.222.0.16 () {56 vars in 1135 bytes} [Fri Jun 24 03:59:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:29.207703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed868ea6-1ea3-4baf-9dc0-37efdd8a3beb None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:29.213738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2f425-0cc7-4957-a8c9-e74829f395e0 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:29.228292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:29.250327 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e1b60-0592-4712-8a63-64497d437077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:29.251244 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e1b60-0592-4712-8a63-64497d437077 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:29.252209 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e1b60-0592-4712-8a63-64497d437077 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:29.252591 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e1b60-0592-4712-8a63-64497d437077 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:29.259911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e1b60-0592-4712-8a63-64497d437077 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:29.262064 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756e1b60-0592-4712-8a63-64497d437077 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:29.277025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-756e1b60-0592-4712-8a63-64497d437077 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:29.277025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:29.277025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:29.277025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:29.277025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:29.277025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:29.277025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:29.277025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:29.277025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:29.277025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:29.277025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:29.277025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:29.277025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:29.277025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:29.277025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:29.278067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:29.278067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:29.278067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:29.278067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:29.278067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:29.278067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:29.278067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:29.278067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:29.278067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:29.278067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:29.278067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:29.278067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:29.278067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:29.278067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:29.278067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:29.278067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:29.278067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:29.279017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:29.279017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:29.279017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:29.279017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:29.279017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:29.279017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:29.279017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:29.279017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:29.283659 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 409/823] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:29.284545 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e1b60-0592-4712-8a63-64497d437077 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:29.324732 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed868ea6-1ea3-4baf-9dc0-37efdd8a3beb None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:29.336976 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:29.363739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327ab3b-e82f-42a0-ac4c-6f2b8f77cea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:29.364917 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327ab3b-e82f-42a0-ac4c-6f2b8f77cea1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:29.365458 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327ab3b-e82f-42a0-ac4c-6f2b8f77cea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:29.365943 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327ab3b-e82f-42a0-ac4c-6f2b8f77cea1 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:29.367004 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327ab3b-e82f-42a0-ac4c-6f2b8f77cea1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:29.367543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327ab3b-e82f-42a0-ac4c-6f2b8f77cea1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:29.370413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327ab3b-e82f-42a0-ac4c-6f2b8f77cea1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:29.376335 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 415/824] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jun 24 03:59:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:29.377965 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327ab3b-e82f-42a0-ac4c-6f2b8f77cea1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:29.392102 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e1b60-0592-4712-8a63-64497d437077 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:29.430080 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205c9ab-9562-4eee-af19-c5cdd948fc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:29.430924 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205c9ab-9562-4eee-af19-c5cdd948fc23 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:29.431210 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205c9ab-9562-4eee-af19-c5cdd948fc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:29.431490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4205c9ab-9562-4eee-af19-c5cdd948fc23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:29.442984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205c9ab-9562-4eee-af19-c5cdd948fc23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:29.444451 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4205c9ab-9562-4eee-af19-c5cdd948fc23 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:29.461934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-4205c9ab-9562-4eee-af19-c5cdd948fc23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:29.461934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:29.461934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:29.461934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:29.461934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:29.461934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:29.461934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:29.461934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:29.461934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:29.461934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:29.461934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:29.461934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:29.461934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:29.461934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:29.461934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:29.462896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:29.462896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:29.462896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:29.462896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:29.462896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:29.462896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:29.462896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:29.462896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:29.462896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:29.462896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:29.462896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:29.462896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:29.462896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:29.462896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:29.462896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:29.462896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:29.462896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:29.463804 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:29.463804 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:29.463804 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:29.463804 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:29.463804 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 410/825] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:29.464095 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205c9ab-9562-4eee-af19-c5cdd948fc23 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:29.490914 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327ab3b-e82f-42a0-ac4c-6f2b8f77cea1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:29.497957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:29.525371 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6e8a1-df09-4bca-a9ef-df71ee6f57f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:29.525973 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6e8a1-df09-4bca-a9ef-df71ee6f57f6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:29.526262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6e8a1-df09-4bca-a9ef-df71ee6f57f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:29.526576 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6e8a1-df09-4bca-a9ef-df71ee6f57f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:29.527386 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6e8a1-df09-4bca-a9ef-df71ee6f57f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:29.527693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6e8a1-df09-4bca-a9ef-df71ee6f57f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:29.529218 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6e8a1-df09-4bca-a9ef-df71ee6f57f6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:29.541225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 416/826] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:29.542573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2793d0d-710b-4b17-910f-f3138c2e74b9 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:29.542866 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2793d0d-710b-4b17-910f-f3138c2e74b9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:29.543147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2793d0d-710b-4b17-910f-f3138c2e74b9 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:29.544534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 417/827] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:29.550119 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2793d0d-710b-4b17-910f-f3138c2e74b9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:29.584541 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4205c9ab-9562-4eee-af19-c5cdd948fc23 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:29.585221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:29.622887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ea4b7-c430-4d71-a957-bacd8821def4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:29.623779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ea4b7-c430-4d71-a957-bacd8821def4 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:29.624206 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ea4b7-c430-4d71-a957-bacd8821def4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:29.624482 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ea4b7-c430-4d71-a957-bacd8821def4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:29.625223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ea4b7-c430-4d71-a957-bacd8821def4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:29.627184 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52ea4b7-c430-4d71-a957-bacd8821def4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:29.655425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2793d0d-710b-4b17-910f-f3138c2e74b9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:29.691271 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1abeed-9b80-4af0-bce2-d6fe371223c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:29.691975 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1abeed-9b80-4af0-bce2-d6fe371223c3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:29.692263 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1abeed-9b80-4af0-bce2-d6fe371223c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:29.692544 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1abeed-9b80-4af0-bce2-d6fe371223c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:29.693269 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1abeed-9b80-4af0-bce2-d6fe371223c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:29.693582 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1abeed-9b80-4af0-bce2-d6fe371223c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:29.695045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1abeed-9b80-4af0-bce2-d6fe371223c3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:29.711363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 418/828] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:29.711363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1abeed-9b80-4af0-bce2-d6fe371223c3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:29.718733 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 411/829] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Jun 24 03:59:29 2022] POST /identity/v3/endpoints => generated 327 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:59:29.720889 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983f002-d11c-4a95-b093-3752b791d810 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:29.721245 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983f002-d11c-4a95-b093-3752b791d810 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:29.721606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983f002-d11c-4a95-b093-3752b791d810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:29.741023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3983f002-d11c-4a95-b093-3752b791d810 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:29.742733 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3983f002-d11c-4a95-b093-3752b791d810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:29.832907 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 412/830] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:29.833993 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983f002-d11c-4a95-b093-3752b791d810 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:29.841484 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:29.853267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1abeed-9b80-4af0-bce2-d6fe371223c3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:29.855975 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:29.894628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6cfc6-1505-4945-9fa9-3e99f1f25b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:29.896333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6cfc6-1505-4945-9fa9-3e99f1f25b00 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:29.896333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6cfc6-1505-4945-9fa9-3e99f1f25b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:29.896333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6cfc6-1505-4945-9fa9-3e99f1f25b00 None admin] PATH_INFO: `/v3/projects/68d376986e854a4bb334676572340b4c/users/98f58f91d17949198e66883d234a9e40/roles/3073b5cac26f43089c9ed8c6a5102cff` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:29.913922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d6cfc6-1505-4945-9fa9-3e99f1f25b00 None admin] RBAC: Authorizing `identity:create_grant(project_id=68d376986e854a4bb334676572340b4c, user_id=98f58f91d17949198e66883d234a9e40, role_id=3073b5cac26f43089c9ed8c6a5102cff)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:29.915478 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d6cfc6-1505-4945-9fa9-3e99f1f25b00 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:29.984438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 419/831] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 03:59:29 2022] PUT /identity/v3/projects/68d376986e854a4bb334676572340b4c/users/98f58f91d17949198e66883d234a9e40/roles/3073b5cac26f43089c9ed8c6a5102cff => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:59:29.985278 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6cfc6-1505-4945-9fa9-3e99f1f25b00 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:29.986713 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983f002-d11c-4a95-b093-3752b791d810 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:30.036815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd0142-a5a4-4629-af52-f2176cd7e6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:30.043060 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd0142-a5a4-4629-af52-f2176cd7e6ff None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:30.043350 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd0142-a5a4-4629-af52-f2176cd7e6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:30.043636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd0142-a5a4-4629-af52-f2176cd7e6ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:30.053547 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd0142-a5a4-4629-af52-f2176cd7e6ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:30.055072 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd0142-a5a4-4629-af52-f2176cd7e6ff None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:30.070735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-0ebd0142-a5a4-4629-af52-f2176cd7e6ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:30.070735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:30.070735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:30.070735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:30.070735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:30.070735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:30.070735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:30.070735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:30.070735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:30.070735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:30.070735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:30.070735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:30.070735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:30.070735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:30.070735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:30.071756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:30.071756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:30.071756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:30.071756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:30.071756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:30.071756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:30.071756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:30.071756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:30.071756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:30.071756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:30.071756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:30.071756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:30.071756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:30.071756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:30.071756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:30.071756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:30.071756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:30.073291 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:30.073291 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:30.073291 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:30.073291 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:30.073291 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:30.073291 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:30.073291 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:30.073291 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:30.073291 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 413/832] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:30.077619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd0142-a5a4-4629-af52-f2176cd7e6ff None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:30.127158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6cfc6-1505-4945-9fa9-3e99f1f25b00 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:30.153063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:30.168017 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f1bc1-eae9-4baf-9469-c9b93c3d1850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:30.169189 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f1bc1-eae9-4baf-9469-c9b93c3d1850 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:30.169711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f1bc1-eae9-4baf-9469-c9b93c3d1850 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:30.170196 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f1bc1-eae9-4baf-9469-c9b93c3d1850 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:30.171145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f1bc1-eae9-4baf-9469-c9b93c3d1850 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:30.172877 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f1bc1-eae9-4baf-9469-c9b93c3d1850 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:30.189469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-692f1bc1-eae9-4baf-9469-c9b93c3d1850 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 03:59:30.189469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:30.189469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:30.189469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:30.189469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:30.189469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:30.189469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:30.189469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:30.189469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 03:59:30.189469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:30.189469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:30.189469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:30.189469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:30.189469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:30.189469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:30.189469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:30.189469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:30.190562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:30.190562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:30.190562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:30.190562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:30.190562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:30.190562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:30.190562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:30.190562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:30.190562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:30.190562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:30.190562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:30.190562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:30.190562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:30.190562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:30.190562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:30.190562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:30.191425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:30.191425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:30.191425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:30.191425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:30.191425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:30.191425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:30.191425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:30.191425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:30.191425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:30.191425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:30.191425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:30.191425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:30.191425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 03:59:30.191425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:30.194261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 420/833] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:59:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:30.197468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd0142-a5a4-4629-af52-f2176cd7e6ff None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:30.199625 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f1bc1-eae9-4baf-9469-c9b93c3d1850 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:30.212095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:30.242940 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfb7bc-3724-412d-a195-2fc1a9b34b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:30.244550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfb7bc-3724-412d-a195-2fc1a9b34b1d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:30.245005 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfb7bc-3724-412d-a195-2fc1a9b34b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:30.245427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfb7bc-3724-412d-a195-2fc1a9b34b1d None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:30.246786 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bfb7bc-3724-412d-a195-2fc1a9b34b1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:30.247250 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bfb7bc-3724-412d-a195-2fc1a9b34b1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:30.249443 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bfb7bc-3724-412d-a195-2fc1a9b34b1d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:30.266182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 414/834] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:30.272536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfb7bc-3724-412d-a195-2fc1a9b34b1d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:30.323520 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f1bc1-eae9-4baf-9469-c9b93c3d1850 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:30.370157 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aaa716-ca5e-4cfc-938c-eadacfca68c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:30.371085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aaa716-ca5e-4cfc-938c-eadacfca68c5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:30.371518 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aaa716-ca5e-4cfc-938c-eadacfca68c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:30.371808 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aaa716-ca5e-4cfc-938c-eadacfca68c5 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:30.372537 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aaa716-ca5e-4cfc-938c-eadacfca68c5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:30.372853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aaa716-ca5e-4cfc-938c-eadacfca68c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:30.374760 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aaa716-ca5e-4cfc-938c-eadacfca68c5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:30.380396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 421/835] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:59:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:30.381607 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493b9f8-fad6-4376-bec9-d97a79e4ac4c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:30.381901 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493b9f8-fad6-4376-bec9-d97a79e4ac4c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:30.382183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493b9f8-fad6-4376-bec9-d97a79e4ac4c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:30.383710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 422/836] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:30.386852 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4493b9f8-fad6-4376-bec9-d97a79e4ac4c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:30.394478 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfb7bc-3724-412d-a195-2fc1a9b34b1d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:30.426135 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:30.435774 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4b4e0-a477-4b78-ad82-9266cdc563c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:30.440247 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4b4e0-a477-4b78-ad82-9266cdc563c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:30.440247 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4b4e0-a477-4b78-ad82-9266cdc563c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:30.440247 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4b4e0-a477-4b78-ad82-9266cdc563c1 None admin] PATH_INFO: `/v3/projects/68d376986e854a4bb334676572340b4c/users/6d8e1f1e3c7e42cc9fa6f82aaac55129/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:30.440504 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4b4e0-a477-4b78-ad82-9266cdc563c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=68d376986e854a4bb334676572340b4c, user_id=6d8e1f1e3c7e42cc9fa6f82aaac55129, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:30.441806 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4b4e0-a477-4b78-ad82-9266cdc563c1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:30.465399 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:30.496848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4493b9f8-fad6-4376-bec9-d97a79e4ac4c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:30.530396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c42116-733b-4190-8a15-149bab830223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:30.531077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c42116-733b-4190-8a15-149bab830223 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:30.531725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c42116-733b-4190-8a15-149bab830223 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:30.532015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c42116-733b-4190-8a15-149bab830223 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:30.533291 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c42116-733b-4190-8a15-149bab830223 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:30.533615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c42116-733b-4190-8a15-149bab830223 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:30.535034 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c42116-733b-4190-8a15-149bab830223 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:30.537142 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 415/837] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 03:59:30 2022] PUT /identity/v3/projects/68d376986e854a4bb334676572340b4c/users/6d8e1f1e3c7e42cc9fa6f82aaac55129/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 03:59:30.538571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6696d-b4fe-4437-88c3-4bbe8d1ba0cb None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:30.538866 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6696d-b4fe-4437-88c3-4bbe8d1ba0cb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:30.539148 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6696d-b4fe-4437-88c3-4bbe8d1ba0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:30.550128 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 423/838] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:59:30 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:30.580442 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7db6696d-b4fe-4437-88c3-4bbe8d1ba0cb None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:30.581733 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7db6696d-b4fe-4437-88c3-4bbe8d1ba0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:30.702343 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 416/839] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:30.726731 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c42116-733b-4190-8a15-149bab830223 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:30.836563 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c42116-733b-4190-8a15-149bab830223 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:30.872178 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6aa36-29e2-4677-81ed-dda1775d69d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:30.872998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6aa36-29e2-4677-81ed-dda1775d69d1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:30.873287 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6aa36-29e2-4677-81ed-dda1775d69d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:30.873568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6aa36-29e2-4677-81ed-dda1775d69d1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:30.874233 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6aa36-29e2-4677-81ed-dda1775d69d1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:30.875643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6aa36-29e2-4677-81ed-dda1775d69d1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:30.879071 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:30.889543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-97c6aa36-29e2-4677-81ed-dda1775d69d1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 03:59:30.889543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:30.889543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:30.889543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:30.889543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:30.889543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:30.889543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:30.889543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:30.889543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 03:59:30.889543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:30.889543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:30.889543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:30.889543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:30.889543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:30.889543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:30.889543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:30.893377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:30.893377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:30.893377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:30.893377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:30.893377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:30.893377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:30.893377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:30.893377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:30.893377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:30.893377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:30.893377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:30.893377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:30.893377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:30.893377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:30.893377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:30.893377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:30.894239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:30.894239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:30.894239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:30.894239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:30.894239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:30.894239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:30.894239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:30.894239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:30.894239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:30.894239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:30.894239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:30.894239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:30.894239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:30.894239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 03:59:30.894239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:30.894239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 424/840] 10.222.0.16 () {56 vars in 1132 bytes} [Fri Jun 24 03:59:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:30.897629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6696d-b4fe-4437-88c3-4bbe8d1ba0cb None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:31.009759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6696d-b4fe-4437-88c3-4bbe8d1ba0cb None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:31.030528 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:31.052712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893075f-c9c8-47b4-badb-c218b2d8eace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:31.053429 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893075f-c9c8-47b4-badb-c218b2d8eace None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:31.053765 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893075f-c9c8-47b4-badb-c218b2d8eace None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:31.054089 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893075f-c9c8-47b4-badb-c218b2d8eace None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:31.054895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893075f-c9c8-47b4-badb-c218b2d8eace None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:31.055208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893075f-c9c8-47b4-badb-c218b2d8eace None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:31.056582 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893075f-c9c8-47b4-badb-c218b2d8eace None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:31.061554 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 417/841] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jun 24 03:59:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:31.067397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6aa36-29e2-4677-81ed-dda1775d69d1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:31.175772 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6aa36-29e2-4677-81ed-dda1775d69d1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:31.211744 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643e807d-58ef-418c-a831-6fd461cd2ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:31.212442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e807d-58ef-418c-a831-6fd461cd2ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:31.212731 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e807d-58ef-418c-a831-6fd461cd2ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:31.213015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643e807d-58ef-418c-a831-6fd461cd2ce4 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:31.213753 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e807d-58ef-418c-a831-6fd461cd2ce4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:31.214076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e807d-58ef-418c-a831-6fd461cd2ce4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:31.215544 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643e807d-58ef-418c-a831-6fd461cd2ce4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:31.222002 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 425/842] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jun 24 03:59:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:32.177735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa155e8-395b-46df-b634-4f4b64507b93 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:32.178281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa155e8-395b-46df-b634-4f4b64507b93 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:32.178281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa155e8-395b-46df-b634-4f4b64507b93 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:32.179933 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 418/843] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:32.185092 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c0d99-0b31-46d3-bac4-bfb75f5fbf7f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:32.185492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c0d99-0b31-46d3-bac4-bfb75f5fbf7f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:32.185694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c0d99-0b31-46d3-bac4-bfb75f5fbf7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:32.203424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e22c0d99-0b31-46d3-bac4-bfb75f5fbf7f None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:32.204699 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e22c0d99-0b31-46d3-bac4-bfb75f5fbf7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:32.272192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:32.281371 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 426/844] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 03:59:32.307013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa155e8-395b-46df-b634-4f4b64507b93 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:32.415085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa155e8-395b-46df-b634-4f4b64507b93 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:32.450390 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d5b48-b5c6-4a3f-962f-21f8c4b69183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:32.451190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d5b48-b5c6-4a3f-962f-21f8c4b69183 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:32.451472 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d5b48-b5c6-4a3f-962f-21f8c4b69183 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:32.451745 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d5b48-b5c6-4a3f-962f-21f8c4b69183 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:32.466685 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956d5b48-b5c6-4a3f-962f-21f8c4b69183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:32.475185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-956d5b48-b5c6-4a3f-962f-21f8c4b69183 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:32.475185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:32.475185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:32.475185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:32.475185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:32.475185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:32.475185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:32.475185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:32.475185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:32.475185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:32.475185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:32.475185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:32.475185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:32.475185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:32.475185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:32.475185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:32.476126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:32.476126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:32.476126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:32.476126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:32.476126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:32.476126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:32.476126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:32.476126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:32.476126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:32.476126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:32.476126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:32.476126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:32.476126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:32.476126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:32.476126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:32.476126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:32.476952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:32.476952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:32.476952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:32.476952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:32.476952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:32.476952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:32.476952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:32.476952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:32.476952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:32.476952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:32.476952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 03:59:32.476952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:32.476952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 419/845] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 03:59:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:32.483889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c0d99-0b31-46d3-bac4-bfb75f5fbf7f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:32.592241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22c0d99-0b31-46d3-bac4-bfb75f5fbf7f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:32.629642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858f6bde-493a-403a-bc9c-09d1164efded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:32.630323 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f6bde-493a-403a-bc9c-09d1164efded None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:32.630637 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f6bde-493a-403a-bc9c-09d1164efded None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:32.630918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f6bde-493a-403a-bc9c-09d1164efded None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:32.631669 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f6bde-493a-403a-bc9c-09d1164efded None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:32.631983 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f6bde-493a-403a-bc9c-09d1164efded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:32.633363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858f6bde-493a-403a-bc9c-09d1164efded None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:32.642312 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 427/846] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 03:59:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:32.648124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d5b48-b5c6-4a3f-962f-21f8c4b69183 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:32.694728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:32.749219 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e3bf3-8f68-496c-ade0-6b95d04791ee None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:32.749551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e3bf3-8f68-496c-ade0-6b95d04791ee None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:32.749797 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e3bf3-8f68-496c-ade0-6b95d04791ee None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:32.751671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 428/847] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:32.756692 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457b972-f4bb-4155-b358-0bfe6daac04b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:32.756989 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457b972-f4bb-4155-b358-0bfe6daac04b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:32.757268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457b972-f4bb-4155-b358-0bfe6daac04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:32.760865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d5b48-b5c6-4a3f-962f-21f8c4b69183 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:32.774632 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b457b972-f4bb-4155-b358-0bfe6daac04b None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:32.775933 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b457b972-f4bb-4155-b358-0bfe6daac04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:32.799076 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204ee928-71d1-4e3e-b026-6b8a083c6008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:32.799875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ee928-71d1-4e3e-b026-6b8a083c6008 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:32.800159 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ee928-71d1-4e3e-b026-6b8a083c6008 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:32.800433 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ee928-71d1-4e3e-b026-6b8a083c6008 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:32.811460 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ee928-71d1-4e3e-b026-6b8a083c6008 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:32.812890 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204ee928-71d1-4e3e-b026-6b8a083c6008 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:32.832223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-204ee928-71d1-4e3e-b026-6b8a083c6008 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:32.832223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:32.832223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:32.832223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:32.832223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:32.832223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:32.832223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:32.832223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:32.832223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:32.832223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:32.832223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:32.832223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:32.832223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:32.832223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:32.832223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:32.833162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:32.833162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:32.833162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:32.833162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:32.833162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:32.833162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:32.833162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:32.833162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:32.833162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:32.833162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:32.833162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:32.833162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:32.833162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:32.833162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:32.833162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:32.833162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:32.833162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:32.834110 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:32.834110 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:32.834110 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:32.834110 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:32.834110 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 420/848] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:32.834898 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf9f21-19df-4e4b-9a50-3704e3a956af None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:32.835249 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf9f21-19df-4e4b-9a50-3704e3a956af None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:32.835465 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcf9f21-19df-4e4b-9a50-3704e3a956af None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:32.835890 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:32.836882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 421/849] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:32.839196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf9f21-19df-4e4b-9a50-3704e3a956af None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:32.858626 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 429/850] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:32.858626 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2e8ad-3ef0-4a7e-aa00-2b9a08a2e88d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:32.858626 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2e8ad-3ef0-4a7e-aa00-2b9a08a2e88d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:32.858626 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e2e8ad-3ef0-4a7e-aa00-2b9a08a2e88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:32.879933 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a7e2e8ad-3ef0-4a7e-aa00-2b9a08a2e88d None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:32.881327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a7e2e8ad-3ef0-4a7e-aa00-2b9a08a2e88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:32.948418 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:32.953112 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 430/851] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:32.954077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2e8ad-3ef0-4a7e-aa00-2b9a08a2e88d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:32.965935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcf9f21-19df-4e4b-9a50-3704e3a956af None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:33.007342 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11724b-5a57-4bae-9811-a4b8032f4216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:33.008500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11724b-5a57-4bae-9811-a4b8032f4216 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:33.008728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11724b-5a57-4bae-9811-a4b8032f4216 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:33.009018 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c11724b-5a57-4bae-9811-a4b8032f4216 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:33.009757 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11724b-5a57-4bae-9811-a4b8032f4216 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:33.010122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11724b-5a57-4bae-9811-a4b8032f4216 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:33.012104 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c11724b-5a57-4bae-9811-a4b8032f4216 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:33.034592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 422/852] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:33.035792 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11724b-5a57-4bae-9811-a4b8032f4216 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:33.078041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e2e8ad-3ef0-4a7e-aa00-2b9a08a2e88d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:33.095620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:33.114165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876b372-ee66-4d43-abfb-cd9e7cc9d2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:33.114806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876b372-ee66-4d43-abfb-cd9e7cc9d2d7 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:33.115090 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876b372-ee66-4d43-abfb-cd9e7cc9d2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:33.115367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876b372-ee66-4d43-abfb-cd9e7cc9d2d7 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:33.116103 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876b372-ee66-4d43-abfb-cd9e7cc9d2d7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:33.117524 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0876b372-ee66-4d43-abfb-cd9e7cc9d2d7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:33.145896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c11724b-5a57-4bae-9811-a4b8032f4216 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:33.169685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 431/853] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jun 24 03:59:32 2022] POST /identity/v3/services => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:59:33.170554 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876b372-ee66-4d43-abfb-cd9e7cc9d2d7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:33.196143 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b5edf-98db-4f11-8a80-ecddec6c4fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:33.197910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b5edf-98db-4f11-8a80-ecddec6c4fff None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:33.198480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b5edf-98db-4f11-8a80-ecddec6c4fff None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:33.198964 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b5edf-98db-4f11-8a80-ecddec6c4fff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:33.214564 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b5edf-98db-4f11-8a80-ecddec6c4fff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:33.224315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-932b5edf-98db-4f11-8a80-ecddec6c4fff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:59:33.224315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:33.224315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:33.224315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:33.224315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:33.224315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:33.224315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:33.224315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:33.224315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:33.224315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:33.224315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:33.224315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:33.224315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:33.224315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:33.224315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:33.224315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:33.225662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:33.225662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:33.225662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:33.225662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:33.225662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:33.225662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:33.225662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:33.225662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:33.225662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:33.225662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:33.225662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:33.225662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:33.225662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:33.225662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:33.225662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:33.225662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:33.226592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:33.226592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:33.226592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:33.226592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:33.226592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:33.226592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:33.226592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:33.226592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:33.226592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:33.226592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:33.226592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 03:59:33.226592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:33.230826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 423/854] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:59:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:33.237250 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b5edf-98db-4f11-8a80-ecddec6c4fff None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:33.257579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:33.289569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0876b372-ee66-4d43-abfb-cd9e7cc9d2d7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:33.337176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869c30a-74a2-4ab8-bf10-360150f5481c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:33.338030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869c30a-74a2-4ab8-bf10-360150f5481c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:33.338318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869c30a-74a2-4ab8-bf10-360150f5481c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:33.338656 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869c30a-74a2-4ab8-bf10-360150f5481c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:33.348199 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869c30a-74a2-4ab8-bf10-360150f5481c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:33.349559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869c30a-74a2-4ab8-bf10-360150f5481c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:33.365047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-b869c30a-74a2-4ab8-bf10-360150f5481c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 03:59:33.365047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:33.365047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:33.365047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:33.365047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:33.365047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:33.365047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:33.365047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:33.365047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:33.365047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:33.365047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:33.365047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:33.365047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:33.365047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:33.365047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:33.365047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:33.366011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:33.366011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:33.366011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:33.366011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:33.366011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:33.366011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:33.366011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:33.366011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:33.366011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:33.366011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:33.366011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:33.366011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:33.366011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:33.366011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:33.366011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:33.366011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:33.366011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:33.366011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:33.367024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:33.367024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:33.367024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:33.367024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:33.367024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:33.367024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:33.367024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:33.367024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:33.367024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 03:59:33.367024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:33.367024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 432/855] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jun 24 03:59:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:33.372173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869c30a-74a2-4ab8-bf10-360150f5481c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:33.376743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b5edf-98db-4f11-8a80-ecddec6c4fff None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:33.412590 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337fdd86-aa10-4f40-b5ab-0ad975b0739b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:33.413262 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fdd86-aa10-4f40-b5ab-0ad975b0739b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:33.413381 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:33.413548 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fdd86-aa10-4f40-b5ab-0ad975b0739b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:33.414062 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fdd86-aa10-4f40-b5ab-0ad975b0739b None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:33.414602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337fdd86-aa10-4f40-b5ab-0ad975b0739b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:33.414916 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337fdd86-aa10-4f40-b5ab-0ad975b0739b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:33.416351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337fdd86-aa10-4f40-b5ab-0ad975b0739b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:33.426114 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 424/856] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:59:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:33.434413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c0b05-218e-4e7c-9e2a-40f0cb62a0a7 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:33.434413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c0b05-218e-4e7c-9e2a-40f0cb62a0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:33.434413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c0b05-218e-4e7c-9e2a-40f0cb62a0a7 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:33.434413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 425/857] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:33.434413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c0b05-218e-4e7c-9e2a-40f0cb62a0a7 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:33.484752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869c30a-74a2-4ab8-bf10-360150f5481c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:33.508152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:33.522251 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767597e1-a93d-437e-bace-6cb7e0fbb5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:33.522983 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767597e1-a93d-437e-bace-6cb7e0fbb5ea None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:33.523272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767597e1-a93d-437e-bace-6cb7e0fbb5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:33.523551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767597e1-a93d-437e-bace-6cb7e0fbb5ea None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:33.524400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767597e1-a93d-437e-bace-6cb7e0fbb5ea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:33.524720 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767597e1-a93d-437e-bace-6cb7e0fbb5ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:33.526222 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767597e1-a93d-437e-bace-6cb7e0fbb5ea None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:33.541287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c0b05-218e-4e7c-9e2a-40f0cb62a0a7 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:33.545944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 433/858] 10.222.0.16 () {56 vars in 1149 bytes} [Fri Jun 24 03:59:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:33.547332 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfccccf-85af-4a4a-a790-a09d9634ff0a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:33.547682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfccccf-85af-4a4a-a790-a09d9634ff0a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:33.547966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfccccf-85af-4a4a-a790-a09d9634ff0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:33.564781 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-fbfccccf-85af-4a4a-a790-a09d9634ff0a None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:33.565970 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fbfccccf-85af-4a4a-a790-a09d9634ff0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:33.583480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee4630-9312-4007-8e45-4e961fe697e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:33.584993 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee4630-9312-4007-8e45-4e961fe697e9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:33.585107 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee4630-9312-4007-8e45-4e961fe697e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:33.585328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee4630-9312-4007-8e45-4e961fe697e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:33.598294 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee4630-9312-4007-8e45-4e961fe697e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:33.599740 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee4630-9312-4007-8e45-4e961fe697e9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:33.617205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-1aee4630-9312-4007-8e45-4e961fe697e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:33.617205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:33.617205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:33.617205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:33.617205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:33.617205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:33.617205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:33.617205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:33.617205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:33.617205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:33.617205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:33.617205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:33.617205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:33.617205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:33.617205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:33.618096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:33.618096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:33.618096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:33.618096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:33.618096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:33.618096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:33.618096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:33.618096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:33.618096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:33.618096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:33.618096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:33.618096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:33.618096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:33.618096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:33.618096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:33.618096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:33.618096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:33.619998 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:33.619998 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:33.619998 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:33.619998 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:33.619998 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 426/859] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:33.619998 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee4630-9312-4007-8e45-4e961fe697e9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:33.644426 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 434/860] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:33 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:33.645871 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfccccf-85af-4a4a-a790-a09d9634ff0a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:33.677413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:33.731106 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee4630-9312-4007-8e45-4e961fe697e9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:33.756888 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:33.763004 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfccccf-85af-4a4a-a790-a09d9634ff0a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:33.768395 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d652266-b00f-4512-bb5b-86aad9972ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:33.770542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d652266-b00f-4512-bb5b-86aad9972ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:33.770542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d652266-b00f-4512-bb5b-86aad9972ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:33.770542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d652266-b00f-4512-bb5b-86aad9972ac1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:33.797652 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6013ba7-0a96-4c64-ab24-5b5f2c1de97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:33.798877 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6013ba7-0a96-4c64-ab24-5b5f2c1de97b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:33.799160 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6013ba7-0a96-4c64-ab24-5b5f2c1de97b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:33.799436 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6013ba7-0a96-4c64-ab24-5b5f2c1de97b None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:33.800158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6013ba7-0a96-4c64-ab24-5b5f2c1de97b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:33.800466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6013ba7-0a96-4c64-ab24-5b5f2c1de97b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:33.801835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6013ba7-0a96-4c64-ab24-5b5f2c1de97b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:33.813478 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 435/861] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:33.814944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6013ba7-0a96-4c64-ab24-5b5f2c1de97b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:33.847005 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d652266-b00f-4512-bb5b-86aad9972ac1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:33.848774 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d652266-b00f-4512-bb5b-86aad9972ac1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:33.898783 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:33.915252 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6013ba7-0a96-4c64-ab24-5b5f2c1de97b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:33.951482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e08c1-cbdb-4fb0-a93f-e701db6c4698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:33.952156 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e08c1-cbdb-4fb0-a93f-e701db6c4698 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:33.952446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e08c1-cbdb-4fb0-a93f-e701db6c4698 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:33.952727 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e08c1-cbdb-4fb0-a93f-e701db6c4698 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:33.953457 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e08c1-cbdb-4fb0-a93f-e701db6c4698 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:33.954910 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e08c1-cbdb-4fb0-a93f-e701db6c4698 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:33.958444 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 427/862] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:59:33 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 03:59:33.959583 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d652266-b00f-4512-bb5b-86aad9972ac1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:33.984608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:34.058259 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 436/863] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jun 24 03:59:33 2022] POST /identity/v3/services => generated 269 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:59:34.059335 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e08c1-cbdb-4fb0-a93f-e701db6c4698 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:34.062956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d652266-b00f-4512-bb5b-86aad9972ac1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:34.110178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a311ce-5e30-47cf-b3ad-929325f0f79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:34.112836 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a311ce-5e30-47cf-b3ad-929325f0f79d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:34.113342 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a311ce-5e30-47cf-b3ad-929325f0f79d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:34.113770 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a311ce-5e30-47cf-b3ad-929325f0f79d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:34.122928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a311ce-5e30-47cf-b3ad-929325f0f79d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:34.124732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a311ce-5e30-47cf-b3ad-929325f0f79d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:34.142597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-62a311ce-5e30-47cf-b3ad-929325f0f79d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 03:59:34.142597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:34.142597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:34.142597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:34.142597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:34.142597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:34.142597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:34.142597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:34.142597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:34.142597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:34.142597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:34.142597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:34.142597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:34.142597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:34.142597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:34.142597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:34.143674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:34.143674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:34.143674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:34.143674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:34.143674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:34.143674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:34.143674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:34.143674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:34.143674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:34.143674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:34.143674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:34.143674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:34.143674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:34.143674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:34.143674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:34.143674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:34.143674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:34.143674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:34.144624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:34.144624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:34.144624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:34.144624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:34.144624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:34.144624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:34.144624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:34.144624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:34.144624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 03:59:34.144624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:34.151830 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 428/864] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 03:59:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:34.152994 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a311ce-5e30-47cf-b3ad-929325f0f79d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:34.201421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e08c1-cbdb-4fb0-a93f-e701db6c4698 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:34.252843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8e769-562e-43e0-8492-060420da98e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:34.253665 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8e769-562e-43e0-8492-060420da98e1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:34.253943 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8e769-562e-43e0-8492-060420da98e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:34.254203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8e769-562e-43e0-8492-060420da98e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:34.267923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8e769-562e-43e0-8492-060420da98e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:34.268717 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:34.270851 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8e769-562e-43e0-8492-060420da98e1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:34.274073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:34.288927 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a311ce-5e30-47cf-b3ad-929325f0f79d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:34.293815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-7ab8e769-562e-43e0-8492-060420da98e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:34.293815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:34.293815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:34.293815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:34.293815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:34.293815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:34.293815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:34.293815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:34.293815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:34.293815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:34.293815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:34.293815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:34.293815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:34.293815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:34.293815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:34.295165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:34.295165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:34.295165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:34.295165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:34.295165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:34.295165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:34.295165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:34.295165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:34.295165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:34.295165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:34.295165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:34.295165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:34.295165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:34.295165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:34.295165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:34.295165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:34.295165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:34.296672 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:34.296672 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:34.296672 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:34.296672 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:34.296672 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 437/865] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:34.301784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8e769-562e-43e0-8492-060420da98e1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:34.329993 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b4cbe3-aadc-482d-a242-a4cea228324b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:34.330799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4cbe3-aadc-482d-a242-a4cea228324b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:34.331131 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4cbe3-aadc-482d-a242-a4cea228324b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:34.331469 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b4cbe3-aadc-482d-a242-a4cea228324b None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:34.332349 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b4cbe3-aadc-482d-a242-a4cea228324b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:34.332665 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b4cbe3-aadc-482d-a242-a4cea228324b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:34.334158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b4cbe3-aadc-482d-a242-a4cea228324b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:34.354764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 429/866] 10.222.0.16 () {56 vars in 1135 bytes} [Fri Jun 24 03:59:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:34.360426 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b4cbe3-aadc-482d-a242-a4cea228324b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:34.410409 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8e769-562e-43e0-8492-060420da98e1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:34.446627 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dbf995-52dd-4e64-8fb6-652055f62d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:34.447302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbf995-52dd-4e64-8fb6-652055f62d94 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:34.447592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbf995-52dd-4e64-8fb6-652055f62d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:34.447875 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dbf995-52dd-4e64-8fb6-652055f62d94 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:34.448611 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dbf995-52dd-4e64-8fb6-652055f62d94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:34.448926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dbf995-52dd-4e64-8fb6-652055f62d94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:34.450308 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dbf995-52dd-4e64-8fb6-652055f62d94 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:34.467144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 438/867] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:34.468161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dbf995-52dd-4e64-8fb6-652055f62d94 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:34.473817 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b4cbe3-aadc-482d-a242-a4cea228324b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:34.513501 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ac9cc-fec3-4d25-b59c-4bf92dba37be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:34.514303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ac9cc-fec3-4d25-b59c-4bf92dba37be None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:34.514641 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ac9cc-fec3-4d25-b59c-4bf92dba37be None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:34.514969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ac9cc-fec3-4d25-b59c-4bf92dba37be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:34.524078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:34.527874 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:34.576255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dbf995-52dd-4e64-8fb6-652055f62d94 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:34.591728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ac9cc-fec3-4d25-b59c-4bf92dba37be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:34.593401 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ac9cc-fec3-4d25-b59c-4bf92dba37be None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:34.620442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd9a0c-dfe2-4dbf-bbf4-7ced5e985692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:34.621243 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd9a0c-dfe2-4dbf-bbf4-7ced5e985692 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:34.621526 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd9a0c-dfe2-4dbf-bbf4-7ced5e985692 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:34.622269 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd9a0c-dfe2-4dbf-bbf4-7ced5e985692 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:34.632187 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd9a0c-dfe2-4dbf-bbf4-7ced5e985692 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:34.634523 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd9a0c-dfe2-4dbf-bbf4-7ced5e985692 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:34.652840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-97cd9a0c-dfe2-4dbf-bbf4-7ced5e985692 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:34.652840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:34.652840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:34.652840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:34.652840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:34.652840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:34.652840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:34.652840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:34.652840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:34.652840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:34.652840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:34.652840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:34.652840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:34.652840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:34.652840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:34.653789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:34.653789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:34.653789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:34.653789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:34.653789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:34.653789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:34.653789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:34.653789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:34.653789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:34.653789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:34.653789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:34.653789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:34.653789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:34.653789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:34.653789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:34.653789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:34.653789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:34.654772 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:34.654772 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:34.654772 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:34.654772 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:34.654772 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:34.654772 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:34.654772 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:34.654772 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:34.656758 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 439/868] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:34.660329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd9a0c-dfe2-4dbf-bbf4-7ced5e985692 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:34.726577 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 430/869] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:59:34 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 03:59:34.727038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ac9cc-fec3-4d25-b59c-4bf92dba37be None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:34.770844 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:34.796885 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd9a0c-dfe2-4dbf-bbf4-7ced5e985692 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:34.817206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:34.834420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ac9cc-fec3-4d25-b59c-4bf92dba37be None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:34.837378 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c87aaa-f15a-4181-84b2-31dc2e2618a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:34.838280 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c87aaa-f15a-4181-84b2-31dc2e2618a7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:34.838796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c87aaa-f15a-4181-84b2-31dc2e2618a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:34.839299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c87aaa-f15a-4181-84b2-31dc2e2618a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:34.840301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c87aaa-f15a-4181-84b2-31dc2e2618a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:34.840835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c87aaa-f15a-4181-84b2-31dc2e2618a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:34.842622 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c87aaa-f15a-4181-84b2-31dc2e2618a7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:34.857372 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 440/870] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:34.863267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c87aaa-f15a-4181-84b2-31dc2e2618a7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:34.875349 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd947c-9508-4ad7-9222-f57dee51f27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:34.876696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd947c-9508-4ad7-9222-f57dee51f27d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:34.877014 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd947c-9508-4ad7-9222-f57dee51f27d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:34.877296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd947c-9508-4ad7-9222-f57dee51f27d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:34.891453 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd947c-9508-4ad7-9222-f57dee51f27d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:34.892981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd947c-9508-4ad7-9222-f57dee51f27d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:34.915003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-08dd947c-9508-4ad7-9222-f57dee51f27d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:34.915003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:34.915003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:34.915003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:34.915003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:34.915003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:34.915003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:34.915003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:34.915003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:34.915003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:34.915003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:34.915003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:34.915003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:34.915003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:34.915003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:34.915980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:34.915980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:34.915980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:34.915980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:34.915980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:34.915980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:34.915980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:34.915980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:34.915980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:34.915980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:34.915980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:34.915980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:34.915980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:34.915980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:34.915980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:34.915980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:34.915980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:34.916897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:34.916897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:34.916897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:34.916897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:34.920379 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 431/871] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:34.921307 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd947c-9508-4ad7-9222-f57dee51f27d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:35.003147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c87aaa-f15a-4181-84b2-31dc2e2618a7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:35.029749 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:35.047406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd947c-9508-4ad7-9222-f57dee51f27d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:35.051359 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b58f4-7af0-4452-a11e-248ceac34dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:35.052273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b58f4-7af0-4452-a11e-248ceac34dfd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:35.052782 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b58f4-7af0-4452-a11e-248ceac34dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:35.053284 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b58f4-7af0-4452-a11e-248ceac34dfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:35.054471 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b58f4-7af0-4452-a11e-248ceac34dfd None admin] RBAC: Adding query filter params (role.id=3073b5cac26f43089c9ed8c6a5102cff, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=98f58f91d17949198e66883d234a9e40) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:35.055041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b58f4-7af0-4452-a11e-248ceac34dfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:35.056688 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b58f4-7af0-4452-a11e-248ceac34dfd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:35.067231 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 441/872] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:59:34 2022] GET /identity/v3/role_assignments?user.id=98f58f91d17949198e66883d234a9e40&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=3073b5cac26f43089c9ed8c6a5102cff => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:35.090439 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d0521-04ee-4b83-b339-7951b9358622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:35.091540 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d0521-04ee-4b83-b339-7951b9358622 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:35.092033 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d0521-04ee-4b83-b339-7951b9358622 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:35.092517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d0521-04ee-4b83-b339-7951b9358622 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:35.093485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d0521-04ee-4b83-b339-7951b9358622 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:35.094027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d0521-04ee-4b83-b339-7951b9358622 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:35.095957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d0521-04ee-4b83-b339-7951b9358622 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:35.111325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 432/873] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:35.116688 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b58f4-7af0-4452-a11e-248ceac34dfd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:35.168721 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:35.234542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b58f4-7af0-4452-a11e-248ceac34dfd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:35.276480 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19003a9-90f4-453f-a647-9476dc33aa22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:35.277219 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19003a9-90f4-453f-a647-9476dc33aa22 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:35.277509 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19003a9-90f4-453f-a647-9476dc33aa22 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:35.277789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19003a9-90f4-453f-a647-9476dc33aa22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:35.285238 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19003a9-90f4-453f-a647-9476dc33aa22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:35.286754 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19003a9-90f4-453f-a647-9476dc33aa22 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:35.287961 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5fc05-e54f-48b2-a5eb-f183c9412ce4 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:35.288406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5fc05-e54f-48b2-a5eb-f183c9412ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:35.288579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5fc05-e54f-48b2-a5eb-f183c9412ce4 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:35.290102 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 433/874] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:35.295223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e633d7-63aa-4c01-9bcb-64f510aae986 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:35.295513 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e633d7-63aa-4c01-9bcb-64f510aae986 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:35.295794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e633d7-63aa-4c01-9bcb-64f510aae986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:35.302286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-a19003a9-90f4-453f-a647-9476dc33aa22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:35.302286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:35.302286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:35.302286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:35.302286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:35.302286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:35.302286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:35.302286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:35.302286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:35.302286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:35.302286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:35.302286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:35.302286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:35.302286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:35.302286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:35.303540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:35.303540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:35.303540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:35.303540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:35.303540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:35.303540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:35.303540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:35.303540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:35.303540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:35.303540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:35.303540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:35.303540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:35.303540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:35.303540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:35.303540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:35.303540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:35.303540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:35.304517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:35.304517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:35.304517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:35.304517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:35.304517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:35.304517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:35.304517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:35.304517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:35.304517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 442/875] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:35.309667 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19003a9-90f4-453f-a647-9476dc33aa22 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:35.317976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-86e633d7-63aa-4c01-9bcb-64f510aae986 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:35.319203 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-86e633d7-63aa-4c01-9bcb-64f510aae986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:35.401039 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 434/876] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:35.420080 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:35.424033 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19003a9-90f4-453f-a647-9476dc33aa22 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:35.427048 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e633d7-63aa-4c01-9bcb-64f510aae986 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:35.482137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1dbe6-1001-4e1e-8f23-e73d29a452fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:35.483618 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1dbe6-1001-4e1e-8f23-e73d29a452fa None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:35.484127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1dbe6-1001-4e1e-8f23-e73d29a452fa None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:35.484687 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1dbe6-1001-4e1e-8f23-e73d29a452fa None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:35.486634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1dbe6-1001-4e1e-8f23-e73d29a452fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:35.486977 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1dbe6-1001-4e1e-8f23-e73d29a452fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:35.488457 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1dbe6-1001-4e1e-8f23-e73d29a452fa None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:35.501217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 443/877] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:35.507032 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1dbe6-1001-4e1e-8f23-e73d29a452fa None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:35.526314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:35.555769 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e633d7-63aa-4c01-9bcb-64f510aae986 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:35.602703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9213b-951e-4cc2-8551-bc1bf7c57b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:35.603908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9213b-951e-4cc2-8551-bc1bf7c57b74 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:35.604401 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9213b-951e-4cc2-8551-bc1bf7c57b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:35.605447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9213b-951e-4cc2-8551-bc1bf7c57b74 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:35.606618 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9213b-951e-4cc2-8551-bc1bf7c57b74 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:35.608948 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9213b-951e-4cc2-8551-bc1bf7c57b74 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:35.618401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1dbe6-1001-4e1e-8f23-e73d29a452fa None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:35.626850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-14d9213b-951e-4cc2-8551-bc1bf7c57b74 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 03:59:35.626850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:35.626850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:35.626850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:35.626850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:35.626850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:35.626850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:35.626850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:35.626850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 03:59:35.626850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:35.626850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:35.626850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:35.626850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:35.626850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:35.626850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:35.626850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:35.626850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:35.627931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:35.627931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:35.627931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:35.627931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:35.627931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:35.627931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:35.627931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:35.627931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:35.627931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:35.627931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:35.627931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:35.627931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:35.627931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:35.627931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:35.627931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:35.627931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:35.628773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:35.628773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:35.628773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:35.628773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:35.628773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:35.628773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:35.628773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:35.628773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:35.628773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:35.628773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:35.628773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:35.628773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:35.628773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 03:59:35.628773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:35.629523 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 435/878] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:59:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:35.636073 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9213b-951e-4cc2-8551-bc1bf7c57b74 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:35.667554 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a7b5c-b4a0-485e-a2c1-3a6c9fb91130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:35.669256 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a7b5c-b4a0-485e-a2c1-3a6c9fb91130 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:35.670043 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a7b5c-b4a0-485e-a2c1-3a6c9fb91130 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:35.670570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a7b5c-b4a0-485e-a2c1-3a6c9fb91130 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:35.671636 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a7b5c-b4a0-485e-a2c1-3a6c9fb91130 None admin] RBAC: Adding query filter params (role.id=f0f7e28cbe314921afdfb657b163436f, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=6d8e1f1e3c7e42cc9fa6f82aaac55129) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:35.672361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a7b5c-b4a0-485e-a2c1-3a6c9fb91130 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:35.675513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a7b5c-b4a0-485e-a2c1-3a6c9fb91130 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:35.683251 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 444/879] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:59:35 2022] GET /identity/v3/role_assignments?user.id=6d8e1f1e3c7e42cc9fa6f82aaac55129&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=f0f7e28cbe314921afdfb657b163436f => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:35.763016 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9213b-951e-4cc2-8551-bc1bf7c57b74 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:35.802783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52541c-ba5e-4d16-a4d3-794a5e3f1fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:35.803472 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52541c-ba5e-4d16-a4d3-794a5e3f1fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:35.803760 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52541c-ba5e-4d16-a4d3-794a5e3f1fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:35.804042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52541c-ba5e-4d16-a4d3-794a5e3f1fe2 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:35.804778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52541c-ba5e-4d16-a4d3-794a5e3f1fe2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:35.805094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52541c-ba5e-4d16-a4d3-794a5e3f1fe2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:35.806691 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52541c-ba5e-4d16-a4d3-794a5e3f1fe2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:35.812885 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 436/880] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:59:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:35.819628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a7b5c-b4a0-485e-a2c1-3a6c9fb91130 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:35.848707 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:35.928675 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a7b5c-b4a0-485e-a2c1-3a6c9fb91130 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:35.963387 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70848c9-6e2b-4060-bc6f-f098f35d041c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:35.964064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70848c9-6e2b-4060-bc6f-f098f35d041c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:35.964359 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70848c9-6e2b-4060-bc6f-f098f35d041c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:35.964638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70848c9-6e2b-4060-bc6f-f098f35d041c None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:35.965360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70848c9-6e2b-4060-bc6f-f098f35d041c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:35.965673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70848c9-6e2b-4060-bc6f-f098f35d041c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:35.967085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70848c9-6e2b-4060-bc6f-f098f35d041c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:35.972859 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 445/881] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:59:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:35.978589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52541c-ba5e-4d16-a4d3-794a5e3f1fe2 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:35.991043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:36.091000 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52541c-ba5e-4d16-a4d3-794a5e3f1fe2 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:36.126027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aadaecc-b4c0-4a93-9151-cbe78cdda232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:36.127309 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadaecc-b4c0-4a93-9151-cbe78cdda232 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:36.127581 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadaecc-b4c0-4a93-9151-cbe78cdda232 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:36.127811 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aadaecc-b4c0-4a93-9151-cbe78cdda232 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:36.128597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadaecc-b4c0-4a93-9151-cbe78cdda232 None admin] RBAC: Adding query filter params (interface=public, service_id=0437879843c74a5f87f6a9f86189c352, region_id=RegionOne) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:36.128918 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadaecc-b4c0-4a93-9151-cbe78cdda232 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:36.130296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aadaecc-b4c0-4a93-9151-cbe78cdda232 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:36.135958 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 437/882] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Jun 24 03:59:35 2022] GET /identity/v3/endpoints?service_id=0437879843c74a5f87f6a9f86189c352&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:36.141115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70848c9-6e2b-4060-bc6f-f098f35d041c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:36.232128 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:36.238522 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753e609-47f4-45a6-a912-a4d80326ec0f None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:36.238816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753e609-47f4-45a6-a912-a4d80326ec0f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:36.239097 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753e609-47f4-45a6-a912-a4d80326ec0f None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:36.240732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 438/883] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 03:59:36.246141 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65d01b-30d1-4e0c-a138-ac388c1ee2df None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:36.246454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65d01b-30d1-4e0c-a138-ac388c1ee2df None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:36.246734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65d01b-30d1-4e0c-a138-ac388c1ee2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:36.251698 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70848c9-6e2b-4060-bc6f-f098f35d041c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:36.274095 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-df65d01b-30d1-4e0c-a138-ac388c1ee2df None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:36.276191 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-df65d01b-30d1-4e0c-a138-ac388c1ee2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:36.286441 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2c09d-3f6c-489f-a06f-2eb5631630b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:36.287122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2c09d-3f6c-489f-a06f-2eb5631630b5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:36.287413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2c09d-3f6c-489f-a06f-2eb5631630b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:36.287691 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2c09d-3f6c-489f-a06f-2eb5631630b5 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:36.288387 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2c09d-3f6c-489f-a06f-2eb5631630b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:36.289764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2c09d-3f6c-489f-a06f-2eb5631630b5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:36.295947 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 446/884] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Jun 24 03:59:36 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 03:59:36.365337 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 439/885] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:36 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 03:59:36.389680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2c09d-3f6c-489f-a06f-2eb5631630b5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:36.495862 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2c09d-3f6c-489f-a06f-2eb5631630b5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:36.531806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a126a0-7d6a-42c4-bfc1-4f52aad2917a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:36.532653 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a126a0-7d6a-42c4-bfc1-4f52aad2917a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:36.532948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a126a0-7d6a-42c4-bfc1-4f52aad2917a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:36.533230 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a126a0-7d6a-42c4-bfc1-4f52aad2917a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:36.533896 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a126a0-7d6a-42c4-bfc1-4f52aad2917a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:36.535304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a126a0-7d6a-42c4-bfc1-4f52aad2917a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:36.547710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-73a126a0-7d6a-42c4-bfc1-4f52aad2917a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 03:59:36.547710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:36.547710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:36.547710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:36.547710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:36.547710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:36.547710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:36.547710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:36.547710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 03:59:36.547710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:36.547710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:36.547710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:36.547710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:36.547710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:36.547710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:36.547710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:36.550318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:36.550318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:36.550318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:36.550318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:36.550318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:36.550318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:36.550318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:36.550318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:36.550318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:36.550318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:36.550318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:36.550318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:36.550318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:36.550318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:36.550318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:36.550318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:36.552330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:36.552330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:36.552330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:36.552330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:36.552330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:36.552330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:36.552330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:36.552330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:36.552330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:36.552330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:36.552330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:36.552330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:36.552330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:36.552330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 03:59:36.552330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:36.552330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 447/886] 10.222.0.16 () {56 vars in 1132 bytes} [Fri Jun 24 03:59:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:36.559702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65d01b-30d1-4e0c-a138-ac388c1ee2df None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:36.561890 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:36.663458 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65d01b-30d1-4e0c-a138-ac388c1ee2df None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:36.700513 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ec1a5-e96b-40ef-84d0-e2ca17a816bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:36.701189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ec1a5-e96b-40ef-84d0-e2ca17a816bc None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:36.701477 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ec1a5-e96b-40ef-84d0-e2ca17a816bc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:36.701759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ec1a5-e96b-40ef-84d0-e2ca17a816bc None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:36.702503 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ec1a5-e96b-40ef-84d0-e2ca17a816bc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:36.702817 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ec1a5-e96b-40ef-84d0-e2ca17a816bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:36.704700 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ec1a5-e96b-40ef-84d0-e2ca17a816bc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:36.710031 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 440/887] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jun 24 03:59:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:36.715639 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a126a0-7d6a-42c4-bfc1-4f52aad2917a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:36.734988 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:36.820208 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a126a0-7d6a-42c4-bfc1-4f52aad2917a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:36.860695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aaf910-3767-4652-aa7e-0555b1776711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:36.861573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aaf910-3767-4652-aa7e-0555b1776711 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:36.861916 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aaf910-3767-4652-aa7e-0555b1776711 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:36.862213 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aaf910-3767-4652-aa7e-0555b1776711 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:36.863030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aaf910-3767-4652-aa7e-0555b1776711 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:36.863354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aaf910-3767-4652-aa7e-0555b1776711 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:36.864751 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aaf910-3767-4652-aa7e-0555b1776711 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:36.870708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 448/888] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jun 24 03:59:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:36.876723 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ec1a5-e96b-40ef-84d0-e2ca17a816bc None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:36.953836 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:36.993955 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ec1a5-e96b-40ef-84d0-e2ca17a816bc None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:37.030642 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc934408-62c7-48e7-836f-6b911f6e6e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:37.031315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc934408-62c7-48e7-836f-6b911f6e6e91 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:37.031602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc934408-62c7-48e7-836f-6b911f6e6e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:37.031881 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc934408-62c7-48e7-836f-6b911f6e6e91 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:37.033222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc934408-62c7-48e7-836f-6b911f6e6e91 None admin] RBAC: Adding query filter params (interface=public, service_id=d7ec7aaa2f2741439026c1165d0edafd, region_id=RegionOne) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:37.033537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc934408-62c7-48e7-836f-6b911f6e6e91 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:37.034947 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc934408-62c7-48e7-836f-6b911f6e6e91 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:37.040722 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 441/889] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Jun 24 03:59:36 2022] GET /identity/v3/endpoints?service_id=d7ec7aaa2f2741439026c1165d0edafd&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:37.046260 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aaf910-3767-4652-aa7e-0555b1776711 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:37.136808 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:37.156624 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aaf910-3767-4652-aa7e-0555b1776711 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:37.194122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51ac8c-d730-421d-a53f-ced407ecff40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:37.195143 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51ac8c-d730-421d-a53f-ced407ecff40 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:37.195628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51ac8c-d730-421d-a53f-ced407ecff40 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:37.196086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51ac8c-d730-421d-a53f-ced407ecff40 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:37.197014 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51ac8c-d730-421d-a53f-ced407ecff40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:37.198600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51ac8c-d730-421d-a53f-ced407ecff40 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:37.205118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 449/890] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Jun 24 03:59:37 2022] GET /identity/v3/services => generated 1268 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 03:59:37.341655 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c4ecf-6184-4b3a-8303-b0d3a3d5dbaa None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:37.341982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c4ecf-6184-4b3a-8303-b0d3a3d5dbaa None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:37.342211 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c4ecf-6184-4b3a-8303-b0d3a3d5dbaa None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:37.343734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 442/891] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:37.350085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3ee9c-9f38-457d-b685-dce3dc2de641 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:37.350468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3ee9c-9f38-457d-b685-dce3dc2de641 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:37.350758 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3ee9c-9f38-457d-b685-dce3dc2de641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:37.368720 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-cce3ee9c-9f38-457d-b685-dce3dc2de641 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:37.370551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-cce3ee9c-9f38-457d-b685-dce3dc2de641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:37.446629 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 450/892] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:37.470987 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c4ecf-6184-4b3a-8303-b0d3a3d5dbaa None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:37.581824 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c4ecf-6184-4b3a-8303-b0d3a3d5dbaa None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:37.612845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:37.618678 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7942cc-dbdc-4717-b253-43a94e3350a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:37.619834 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7942cc-dbdc-4717-b253-43a94e3350a7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:37.620121 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7942cc-dbdc-4717-b253-43a94e3350a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:37.620404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7942cc-dbdc-4717-b253-43a94e3350a7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:37.632033 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7942cc-dbdc-4717-b253-43a94e3350a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:37.640462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-ab7942cc-dbdc-4717-b253-43a94e3350a7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 03:59:37.640462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:37.640462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:37.640462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:37.640462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:37.640462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:37.640462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:37.640462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:37.640462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:37.640462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:37.640462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:37.640462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:37.640462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:37.640462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:37.640462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:37.640462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:37.641409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:37.641409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:37.641409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:37.641409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:37.641409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:37.641409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:37.641409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:37.641409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:37.641409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:37.641409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:37.641409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:37.641409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:37.641409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:37.641409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:37.641409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:37.641409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:37.642328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:37.642328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:37.642328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:37.642328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:37.642328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:37.642328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:37.642328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:37.642328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:37.642328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:37.642328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:37.642328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 03:59:37.642328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:37.642328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 443/893] 10.222.0.16 () {56 vars in 1120 bytes} [Fri Jun 24 03:59:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:37.648953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3ee9c-9f38-457d-b685-dce3dc2de641 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:37.738859 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:37.757907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3ee9c-9f38-457d-b685-dce3dc2de641 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:37.793761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a6a58-1bc3-4589-9131-691f44eff7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:37.794467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a6a58-1bc3-4589-9131-691f44eff7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:37.794762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a6a58-1bc3-4589-9131-691f44eff7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:37.795042 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a6a58-1bc3-4589-9131-691f44eff7f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:37.795802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a6a58-1bc3-4589-9131-691f44eff7f1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:37.796116 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a6a58-1bc3-4589-9131-691f44eff7f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:37.797553 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a6a58-1bc3-4589-9131-691f44eff7f1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:37.806588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 451/894] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Jun 24 03:59:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:37.815505 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7942cc-dbdc-4717-b253-43a94e3350a7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:37.918090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7942cc-dbdc-4717-b253-43a94e3350a7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:37.949327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991f044-6910-45fc-90b4-ed1b23d6114a None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:37.949659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991f044-6910-45fc-90b4-ed1b23d6114a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:37.949903 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991f044-6910-45fc-90b4-ed1b23d6114a None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:37.952546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 452/895] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 03:59:37.956859 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0f25e-77c1-41a4-be69-554bdca690e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:37.957596 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8ed20-7d55-4d5d-b45d-d61150f94ca6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:37.957723 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0f25e-77c1-41a4-be69-554bdca690e1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:37.957888 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8ed20-7d55-4d5d-b45d-d61150f94ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:37.957984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0f25e-77c1-41a4-be69-554bdca690e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:37.958163 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8ed20-7d55-4d5d-b45d-d61150f94ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:37.958261 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0f25e-77c1-41a4-be69-554bdca690e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:37.971835 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0f25e-77c1-41a4-be69-554bdca690e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:37.973315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0f25e-77c1-41a4-be69-554bdca690e1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:37.974946 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1df8ed20-7d55-4d5d-b45d-d61150f94ca6 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:37.976332 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1df8ed20-7d55-4d5d-b45d-d61150f94ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:37.995596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-afc0f25e-77c1-41a4-be69-554bdca690e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:37.995596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:37.995596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:37.995596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:37.995596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:37.995596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:37.995596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:37.995596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:37.995596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:37.995596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:37.995596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:37.995596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:37.995596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:37.995596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:37.995596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:37.996515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:37.996515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:37.996515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:37.996515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:37.996515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:37.996515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:37.996515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:37.996515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:37.996515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:37.996515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:37.996515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:37.996515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:37.996515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:37.996515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:37.996515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:37.996515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:37.996515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:37.997418 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:37.997418 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:37.997418 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:37.997418 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:37.997418 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 444/896] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:38.004444 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0f25e-77c1-41a4-be69-554bdca690e1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:38.050997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 453/897] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 03:59:38.074027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8ed20-7d55-4d5d-b45d-d61150f94ca6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:38.076321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:38.145764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0f25e-77c1-41a4-be69-554bdca690e1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:38.184373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8ed20-7d55-4d5d-b45d-d61150f94ca6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:38.187179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:38.191355 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed79490-6701-4132-a596-f24373d9aaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:38.192734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed79490-6701-4132-a596-f24373d9aaf6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:38.193228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed79490-6701-4132-a596-f24373d9aaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:38.194050 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed79490-6701-4132-a596-f24373d9aaf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:38.195032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed79490-6701-4132-a596-f24373d9aaf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:38.195848 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed79490-6701-4132-a596-f24373d9aaf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:38.197923 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed79490-6701-4132-a596-f24373d9aaf6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:38.220547 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 445/898] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:38.221396 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed79490-6701-4132-a596-f24373d9aaf6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:38.226952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c1c97-2bb9-4648-8e82-d8f452116031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:38.227767 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c1c97-2bb9-4648-8e82-d8f452116031 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:38.228051 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c1c97-2bb9-4648-8e82-d8f452116031 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:38.228328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c1c97-2bb9-4648-8e82-d8f452116031 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:38.229080 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c1c97-2bb9-4648-8e82-d8f452116031 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:38.230495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c1c97-2bb9-4648-8e82-d8f452116031 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:38.243380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-d09c1c97-2bb9-4648-8e82-d8f452116031 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 03:59:38.243380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:38.243380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:38.243380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:38.243380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:38.243380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:38.243380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:38.243380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:38.243380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 03:59:38.243380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:38.243380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:38.243380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:38.243380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:38.243380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:38.243380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:38.243380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:38.243380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:38.244678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:38.244678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:38.244678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:38.244678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:38.244678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:38.244678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:38.244678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:38.244678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:38.244678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:38.244678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:38.244678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:38.244678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:38.244678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:38.244678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:38.244678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:38.244678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:38.245578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:38.245578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:38.245578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:38.245578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:38.245578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:38.245578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:38.245578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:38.245578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:38.245578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:38.245578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:38.245578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:38.245578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:38.245578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 03:59:38.245578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:38.250035 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 454/899] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jun 24 03:59:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:38.252103 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c1c97-2bb9-4648-8e82-d8f452116031 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:38.357018 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed79490-6701-4132-a596-f24373d9aaf6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:38.358911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:38.371436 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c1c97-2bb9-4648-8e82-d8f452116031 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:38.402949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e24aa8-3625-463b-89dd-91166999f26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:38.404128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e24aa8-3625-463b-89dd-91166999f26f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:38.404645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e24aa8-3625-463b-89dd-91166999f26f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:38.405122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e24aa8-3625-463b-89dd-91166999f26f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:38.413201 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:38.413887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e24aa8-3625-463b-89dd-91166999f26f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:38.415652 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e24aa8-3625-463b-89dd-91166999f26f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:38.420391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e50829-8560-47f9-8f35-6c612fcb6a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:38.421077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e50829-8560-47f9-8f35-6c612fcb6a41 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:38.421426 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e50829-8560-47f9-8f35-6c612fcb6a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:38.421713 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e50829-8560-47f9-8f35-6c612fcb6a41 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:38.422519 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e50829-8560-47f9-8f35-6c612fcb6a41 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:38.434042 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e50829-8560-47f9-8f35-6c612fcb6a41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:38.434042 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e50829-8560-47f9-8f35-6c612fcb6a41 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:38.434042 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 455/900] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jun 24 03:59:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:38.434301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e3e24aa8-3625-463b-89dd-91166999f26f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 03:59:38.434301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:38.434301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:38.434301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:38.434301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:38.434301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:38.434301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:38.434301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:38.434301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:38.434301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:38.434301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:38.434301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:38.434301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:38.434301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:38.434301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:38.434301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:38.435222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:38.435222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:38.435222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:38.435222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:38.435222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:38.435222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:38.435222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:38.435222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:38.435222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:38.435222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:38.435222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:38.435222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:38.435222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:38.435222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:38.435222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:38.435222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:38.435222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:38.435222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:38.436174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:38.436174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:38.436174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:38.436174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:38.436174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:38.436174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:38.436174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:38.436174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:38.436174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 03:59:38.436174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:38.437883 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e50829-8560-47f9-8f35-6c612fcb6a41 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:38.439121 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 446/901] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jun 24 03:59:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:38.444724 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e24aa8-3625-463b-89dd-91166999f26f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:38.560445 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e50829-8560-47f9-8f35-6c612fcb6a41 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:38.574956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e24aa8-3625-463b-89dd-91166999f26f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:38.594294 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c44ad5-ebe5-46bd-9c7a-2611747dc551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:38.595522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c44ad5-ebe5-46bd-9c7a-2611747dc551 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:38.595812 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c44ad5-ebe5-46bd-9c7a-2611747dc551 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:38.596223 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c44ad5-ebe5-46bd-9c7a-2611747dc551 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:38.596934 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c44ad5-ebe5-46bd-9c7a-2611747dc551 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:38.597244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c44ad5-ebe5-46bd-9c7a-2611747dc551 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:38.598667 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c44ad5-ebe5-46bd-9c7a-2611747dc551 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:38.603050 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 456/902] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jun 24 03:59:38 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:38.604236 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f01d3-d911-4d7f-98a5-85884caf074c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:38.604526 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f01d3-d911-4d7f-98a5-85884caf074c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:38.604805 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f01d3-d911-4d7f-98a5-85884caf074c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:38.606210 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 457/903] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:38.612563 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc63996-1fe8-4d04-a502-ce0e840bf369 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:38.612868 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc63996-1fe8-4d04-a502-ce0e840bf369 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:38.613132 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc63996-1fe8-4d04-a502-ce0e840bf369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:38.621876 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9426588-8e6b-46ea-96c6-380f36342e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:38.623027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9426588-8e6b-46ea-96c6-380f36342e7e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:38.623540 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9426588-8e6b-46ea-96c6-380f36342e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:38.624026 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9426588-8e6b-46ea-96c6-380f36342e7e None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:38.625101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9426588-8e6b-46ea-96c6-380f36342e7e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:38.626474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9426588-8e6b-46ea-96c6-380f36342e7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:38.628677 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9426588-8e6b-46ea-96c6-380f36342e7e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:38.632983 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5dc63996-1fe8-4d04-a502-ce0e840bf369 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:38.633366 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5dc63996-1fe8-4d04-a502-ce0e840bf369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:38.660408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 447/904] 10.222.0.16 () {56 vars in 1149 bytes} [Fri Jun 24 03:59:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:38.665949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9426588-8e6b-46ea-96c6-380f36342e7e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:38.698046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:38.706249 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:38.730741 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 458/905] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:38 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:38.747230 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc63996-1fe8-4d04-a502-ce0e840bf369 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:38.794154 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9426588-8e6b-46ea-96c6-380f36342e7e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:38.840956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce23c8c-1050-4e4b-91c4-93ef2653b347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:38.841536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce23c8c-1050-4e4b-91c4-93ef2653b347 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:38.841823 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce23c8c-1050-4e4b-91c4-93ef2653b347 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:38.842102 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce23c8c-1050-4e4b-91c4-93ef2653b347 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:38.874707 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc63996-1fe8-4d04-a502-ce0e840bf369 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:38.911954 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277e60f-13eb-44cb-9516-0915d09f6440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:38.912770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277e60f-13eb-44cb-9516-0915d09f6440 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:38.913064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277e60f-13eb-44cb-9516-0915d09f6440 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:38.913339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5277e60f-13eb-44cb-9516-0915d09f6440 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:38.914009 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5277e60f-13eb-44cb-9516-0915d09f6440 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:38.915445 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5277e60f-13eb-44cb-9516-0915d09f6440 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:38.921996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce23c8c-1050-4e4b-91c4-93ef2653b347 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:38.923858 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce23c8c-1050-4e4b-91c4-93ef2653b347 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:38.928145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-5277e60f-13eb-44cb-9516-0915d09f6440 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 03:59:38.928145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:38.928145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:38.928145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:38.928145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:38.928145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:38.928145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:38.928145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:38.928145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 03:59:38.928145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:38.928145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:38.928145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:38.928145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:38.928145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:38.928145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:38.928145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:38.928145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:38.929180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:38.929180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:38.929180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:38.929180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:38.929180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:38.929180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:38.929180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:38.929180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:38.929180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:38.929180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:38.929180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:38.929180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:38.929180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:38.929180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:38.929180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:38.929180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:38.930015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:38.930015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:38.930015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:38.930015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:38.930015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:38.930015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:38.930015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:38.930015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:38.930015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:38.930015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:38.930015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:38.930015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:38.930015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 03:59:38.930015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:38.930015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 459/906] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:59:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:38.937499 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277e60f-13eb-44cb-9516-0915d09f6440 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:38.957944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:39.009827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:39.069398 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5277e60f-13eb-44cb-9516-0915d09f6440 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:39.070008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 448/907] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:59:38 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 03:59:39.071259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce23c8c-1050-4e4b-91c4-93ef2653b347 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:39.108770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae191b-8e35-470c-bef5-936cd44949ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:39.109331 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae191b-8e35-470c-bef5-936cd44949ba None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:39.109640 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae191b-8e35-470c-bef5-936cd44949ba None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:39.109892 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae191b-8e35-470c-bef5-936cd44949ba None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:39.110685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae191b-8e35-470c-bef5-936cd44949ba None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:39.110994 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae191b-8e35-470c-bef5-936cd44949ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:39.112376 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae191b-8e35-470c-bef5-936cd44949ba None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:39.122136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 460/908] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:59:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:39.123263 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae191b-8e35-470c-bef5-936cd44949ba None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:39.207551 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce23c8c-1050-4e4b-91c4-93ef2653b347 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:39.229543 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:39.240101 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae191b-8e35-470c-bef5-936cd44949ba None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:39.246207 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441326ef-1967-4082-a9ac-d716c28a93a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:39.247107 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441326ef-1967-4082-a9ac-d716c28a93a6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:39.247482 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441326ef-1967-4082-a9ac-d716c28a93a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:39.247829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441326ef-1967-4082-a9ac-d716c28a93a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:39.258895 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:39.260754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441326ef-1967-4082-a9ac-d716c28a93a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:39.262300 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441326ef-1967-4082-a9ac-d716c28a93a6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:39.279062 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38011c79-b9e5-4533-a5c3-bcd30c757315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:39.280063 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38011c79-b9e5-4533-a5c3-bcd30c757315 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:39.280555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38011c79-b9e5-4533-a5c3-bcd30c757315 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:39.281023 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38011c79-b9e5-4533-a5c3-bcd30c757315 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:39.281982 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38011c79-b9e5-4533-a5c3-bcd30c757315 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:39.283027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38011c79-b9e5-4533-a5c3-bcd30c757315 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:39.284132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-441326ef-1967-4082-a9ac-d716c28a93a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:39.284132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:39.284132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:39.284132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:39.284132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:39.284132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:39.284132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:39.284132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:39.284132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:39.284132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:39.284132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:39.284132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:39.284132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:39.284132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:39.284132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:39.285008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:39.285008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:39.285008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:39.285008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:39.285008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:39.285008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:39.285008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:39.285008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:39.285008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:39.285008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:39.285008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:39.285008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:39.285008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:39.285008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:39.285008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:39.285008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:39.285008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:39.285994 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:39.285994 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:39.285994 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:39.285994 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:39.285994 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 449/909] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:39.287160 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38011c79-b9e5-4533-a5c3-bcd30c757315 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:39.291037 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441326ef-1967-4082-a9ac-d716c28a93a6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:39.293334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 461/910] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Jun 24 03:59:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:39.300115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38011c79-b9e5-4533-a5c3-bcd30c757315 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:39.405097 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441326ef-1967-4082-a9ac-d716c28a93a6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:39.415555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38011c79-b9e5-4533-a5c3-bcd30c757315 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:39.445249 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543b14d-e740-40ed-9a85-b12566393d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:39.446187 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543b14d-e740-40ed-9a85-b12566393d1a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:39.446512 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543b14d-e740-40ed-9a85-b12566393d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:39.446894 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543b14d-e740-40ed-9a85-b12566393d1a None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:39.447708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543b14d-e740-40ed-9a85-b12566393d1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:39.448104 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543b14d-e740-40ed-9a85-b12566393d1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:39.449590 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543b14d-e740-40ed-9a85-b12566393d1a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:39.449750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11ae3f-3f76-4a51-be00-f1927c74be8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:39.450421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11ae3f-3f76-4a51-be00-f1927c74be8b None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:39.450715 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11ae3f-3f76-4a51-be00-f1927c74be8b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:39.450996 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11ae3f-3f76-4a51-be00-f1927c74be8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:39.451737 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11ae3f-3f76-4a51-be00-f1927c74be8b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:39.453088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11ae3f-3f76-4a51-be00-f1927c74be8b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:39.464744 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 450/911] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:39.466161 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dabb0f-6b49-4e19-a26b-4ffb30f746d2 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:39.466558 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dabb0f-6b49-4e19-a26b-4ffb30f746d2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:39.466908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dabb0f-6b49-4e19-a26b-4ffb30f746d2 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:39.468438 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 451/912] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:39.471140 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dabb0f-6b49-4e19-a26b-4ffb30f746d2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:39.502187 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:39.550155 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 462/913] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Jun 24 03:59:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 03:59:39.551832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81bfc25-af37-437c-aeba-44adc3bcc409 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:39.552207 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81bfc25-af37-437c-aeba-44adc3bcc409 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:39.552556 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81bfc25-af37-437c-aeba-44adc3bcc409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:39.573673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a81bfc25-af37-437c-aeba-44adc3bcc409 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:39.575304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a81bfc25-af37-437c-aeba-44adc3bcc409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:39.592573 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dabb0f-6b49-4e19-a26b-4ffb30f746d2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:39.614692 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:39.638485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d8a88-c13f-43df-83a4-93d93562ef61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:39.639381 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d8a88-c13f-43df-83a4-93d93562ef61 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:39.639733 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d8a88-c13f-43df-83a4-93d93562ef61 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:39.640113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d8a88-c13f-43df-83a4-93d93562ef61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:39.651520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d8a88-c13f-43df-83a4-93d93562ef61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:39.653607 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d8a88-c13f-43df-83a4-93d93562ef61 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:39.670172 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 463/914] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:39.675303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-0c9d8a88-c13f-43df-83a4-93d93562ef61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:39.675303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:39.675303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:39.675303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:39.675303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:39.675303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:39.675303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:39.675303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:39.675303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:39.675303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:39.675303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:39.675303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:39.675303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:39.675303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:39.675303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:39.676225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:39.676225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:39.676225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:39.676225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:39.676225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:39.676225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:39.676225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:39.676225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:39.676225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:39.676225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:39.676225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:39.676225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:39.676225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:39.676225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:39.676225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:39.676225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:39.676225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:39.677180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:39.677180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:39.677180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:39.677180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:39.677180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:39.677180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:39.677180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:39.677180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:39.683050 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 452/915] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:39.684447 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81bfc25-af37-437c-aeba-44adc3bcc409 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:39.698609 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d8a88-c13f-43df-83a4-93d93562ef61 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:39.809888 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81bfc25-af37-437c-aeba-44adc3bcc409 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:39.817436 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:39.832768 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d8a88-c13f-43df-83a4-93d93562ef61 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:39.855423 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf87b2-1e07-493b-ac87-a5a928a48701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:39.856093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf87b2-1e07-493b-ac87-a5a928a48701 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:39.856382 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf87b2-1e07-493b-ac87-a5a928a48701 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:39.856664 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf87b2-1e07-493b-ac87-a5a928a48701 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:39.857466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf87b2-1e07-493b-ac87-a5a928a48701 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:39.857779 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf87b2-1e07-493b-ac87-a5a928a48701 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:39.859249 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf87b2-1e07-493b-ac87-a5a928a48701 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:39.878758 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 464/916] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:39.882637 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e18b0-6bd8-453c-91d1-c5d08d6c64d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:39.883737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e18b0-6bd8-453c-91d1-c5d08d6c64d1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:39.884022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e18b0-6bd8-453c-91d1-c5d08d6c64d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:39.884298 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e18b0-6bd8-453c-91d1-c5d08d6c64d1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:39.884956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e18b0-6bd8-453c-91d1-c5d08d6c64d1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:39.886335 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e18b0-6bd8-453c-91d1-c5d08d6c64d1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:39.887352 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf87b2-1e07-493b-ac87-a5a928a48701 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:39.904868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-fe5e18b0-6bd8-453c-91d1-c5d08d6c64d1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 03:59:39.904868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:39.904868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:39.904868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:39.904868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:39.904868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:39.904868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:39.904868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:39.904868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 03:59:39.904868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:39.904868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:39.904868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:39.904868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:39.904868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:39.904868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:39.904868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:39.905911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:39.905911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:39.905911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:39.905911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:39.905911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:39.905911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:39.905911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:39.905911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:39.905911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:39.905911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:39.905911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:39.905911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:39.905911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:39.905911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:39.905911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:39.905911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:39.906846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:39.906846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:39.906846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:39.906846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:39.906846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:39.906846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:39.906846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:39.906846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:39.906846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:39.906846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:39.906846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:39.906846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:39.906846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:39.906846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 03:59:39.906846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:39.906846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 453/917] 10.222.0.16 () {56 vars in 1132 bytes} [Fri Jun 24 03:59:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:39.912164 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e18b0-6bd8-453c-91d1-c5d08d6c64d1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:39.922244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:40.002434 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf87b2-1e07-493b-ac87-a5a928a48701 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:40.052254 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b18d7-4491-4cb8-865b-9a9b67e880cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:40.052921 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b18d7-4491-4cb8-865b-9a9b67e880cc None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:40.053208 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b18d7-4491-4cb8-865b-9a9b67e880cc None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:40.053487 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1b18d7-4491-4cb8-865b-9a9b67e880cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:40.054334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b18d7-4491-4cb8-865b-9a9b67e880cc None admin] RBAC: Adding query filter params (role.id=254f32a34d7b4de7b5c039848b8153d6, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=98f58f91d17949198e66883d234a9e40) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:40.054684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b18d7-4491-4cb8-865b-9a9b67e880cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:40.056126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1b18d7-4491-4cb8-865b-9a9b67e880cc None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:40.059550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e18b0-6bd8-453c-91d1-c5d08d6c64d1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:40.066686 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 465/918] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:59:39 2022] GET /identity/v3/role_assignments?user.id=98f58f91d17949198e66883d234a9e40&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=254f32a34d7b4de7b5c039848b8153d6 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:40.100939 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0128e1b-217d-480d-b5d7-d5ee13462d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:40.102052 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0128e1b-217d-480d-b5d7-d5ee13462d67 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:40.102643 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0128e1b-217d-480d-b5d7-d5ee13462d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:40.103121 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0128e1b-217d-480d-b5d7-d5ee13462d67 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:40.104059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0128e1b-217d-480d-b5d7-d5ee13462d67 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:40.104622 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0128e1b-217d-480d-b5d7-d5ee13462d67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:40.106402 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0128e1b-217d-480d-b5d7-d5ee13462d67 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:40.113458 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 454/919] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jun 24 03:59:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:40.120175 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b18d7-4491-4cb8-865b-9a9b67e880cc None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:40.224980 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:40.234878 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1b18d7-4491-4cb8-865b-9a9b67e880cc None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:40.271208 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae83d8f-5d25-4ebf-8220-dff59c6c8774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:40.272248 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae83d8f-5d25-4ebf-8220-dff59c6c8774 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:40.272535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae83d8f-5d25-4ebf-8220-dff59c6c8774 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:40.272812 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae83d8f-5d25-4ebf-8220-dff59c6c8774 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:40.273529 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae83d8f-5d25-4ebf-8220-dff59c6c8774 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:40.273842 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae83d8f-5d25-4ebf-8220-dff59c6c8774 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:40.275244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae83d8f-5d25-4ebf-8220-dff59c6c8774 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:40.280600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 466/920] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jun 24 03:59:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:40.287481 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0128e1b-217d-480d-b5d7-d5ee13462d67 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:40.308780 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:40.396172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0128e1b-217d-480d-b5d7-d5ee13462d67 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:40.431299 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86248f5c-103b-4557-91e4-4d178130d949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:40.431960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86248f5c-103b-4557-91e4-4d178130d949 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:40.432245 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86248f5c-103b-4557-91e4-4d178130d949 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:40.432522 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86248f5c-103b-4557-91e4-4d178130d949 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:40.433267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86248f5c-103b-4557-91e4-4d178130d949 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:40.434671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86248f5c-103b-4557-91e4-4d178130d949 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:40.546501 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 455/921] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Jun 24 03:59:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:59:40.708274 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ee03a-b5dc-4e8a-9435-ce0f9a7db425 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:40.708781 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ee03a-b5dc-4e8a-9435-ce0f9a7db425 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:40.708878 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ee03a-b5dc-4e8a-9435-ce0f9a7db425 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:40.710246 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 467/922] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:40.715747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24d747-bdfd-405e-8c5e-ceda93c39242 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:40.716598 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24d747-bdfd-405e-8c5e-ceda93c39242 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:40.717086 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24d747-bdfd-405e-8c5e-ceda93c39242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:40.737257 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3e24d747-bdfd-405e-8c5e-ceda93c39242 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:40.738505 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3e24d747-bdfd-405e-8c5e-ceda93c39242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:40.814344 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:40.820771 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 456/923] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:40.844334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ee03a-b5dc-4e8a-9435-ce0f9a7db425 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:40.956522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ee03a-b5dc-4e8a-9435-ce0f9a7db425 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:40.991108 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30e3f0-5fa9-449b-8ff3-f54b9d6525af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:40.991838 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30e3f0-5fa9-449b-8ff3-f54b9d6525af None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:40.992156 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30e3f0-5fa9-449b-8ff3-f54b9d6525af None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:40.992435 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30e3f0-5fa9-449b-8ff3-f54b9d6525af None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:40.993275 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30e3f0-5fa9-449b-8ff3-f54b9d6525af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:40.994932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30e3f0-5fa9-449b-8ff3-f54b9d6525af None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:41.080842 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 468/924] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jun 24 03:59:40 2022] POST /identity/v3/services => generated 241 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:59:41.815233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910bede-af59-4e7f-b6ac-ec880a9da8be None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:41.815758 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910bede-af59-4e7f-b6ac-ec880a9da8be None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:41.815758 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910bede-af59-4e7f-b6ac-ec880a9da8be None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:41.817443 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 457/925] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:41.822616 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f7cdc-65c2-4aa9-bb62-0e73d6e97e6a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:41.823043 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f7cdc-65c2-4aa9-bb62-0e73d6e97e6a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:41.823217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f7cdc-65c2-4aa9-bb62-0e73d6e97e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:41.840254 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-de3f7cdc-65c2-4aa9-bb62-0e73d6e97e6a None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:41.841440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-de3f7cdc-65c2-4aa9-bb62-0e73d6e97e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:41.851545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:41.918359 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 469/926] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:41.942912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910bede-af59-4e7f-b6ac-ec880a9da8be None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:42.064140 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910bede-af59-4e7f-b6ac-ec880a9da8be None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:42.100553 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66180e-0e31-438e-8878-7f293519cefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:42.101139 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66180e-0e31-438e-8878-7f293519cefd None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:42.101426 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66180e-0e31-438e-8878-7f293519cefd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:42.101703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66180e-0e31-438e-8878-7f293519cefd None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:42.102455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66180e-0e31-438e-8878-7f293519cefd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:42.104028 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66180e-0e31-438e-8878-7f293519cefd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:42.105407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: WARNING keystone.server.flask.common [None req-ab66180e-0e31-438e-8878-7f293519cefd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 24 03:59:42.158189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.sql.core [None req-ab66180e-0e31-438e-8878-7f293519cefd None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 03:59:42.158189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 03:59:42.158189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [parameters: {'id': '34e83c648a56401b80ca563bfa371211', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 03:59:42.158189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80600) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 03:59:42.176663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-ab66180e-0e31-438e-8878-7f293519cefd None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 34e83c648a56401b80ca563bfa371211.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 34e83c648a56401b80ca563bfa371211. Jun 24 03:59:42.176663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:42.176663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 03:59:42.176663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 03:59:42.176663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 03:59:42.176663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 03:59:42.176663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 03:59:42.176663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 03:59:42.176663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 03:59:42.176663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application conn.query(q) Jun 24 03:59:42.176663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 03:59:42.176663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 03:59:42.176663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 03:59:42.177544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result.read() Jun 24 03:59:42.177544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 03:59:42.177544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 03:59:42.177544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 03:59:42.177544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 03:59:42.177544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 03:59:42.177544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 03:59:42.177544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 03:59:42.177544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 03:59:42.177544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 03:59:42.177544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:42.177544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 03:59:42.177544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:42.177544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:42.177544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 03:59:42.177544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 03:59:42.177544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 24 03:59:42.177544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 24 03:59:42.179091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 03:59:42.179091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application next(self.gen) Jun 24 03:59:42.179091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 03:59:42.179091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application yield resource Jun 24 03:59:42.179091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 03:59:42.179091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application next(self.gen) Jun 24 03:59:42.179091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 03:59:42.179091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 03:59:42.179091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 03:59:42.179091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 03:59:42.179091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 03:59:42.179091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise self.value Jun 24 03:59:42.179091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 03:59:42.179091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 03:59:42.179091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 03:59:42.179091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application session.commit() Jun 24 03:59:42.179091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 03:59:42.180640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 03:59:42.180640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 03:59:42.180640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 03:59:42.180640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 03:59:42.180640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.session.flush() Jun 24 03:59:42.180640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 03:59:42.180640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 03:59:42.180640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 03:59:42.180640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 03:59:42.180640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 03:59:42.180640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application compat.raise_( Jun 24 03:59:42.180640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 03:59:42.180640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception Jun 24 03:59:42.180640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 03:59:42.180640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 03:59:42.180640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 03:59:42.180640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 03:59:42.182152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 03:59:42.182152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 03:59:42.182152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 03:59:42.182152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 03:59:42.182152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 03:59:42.182152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 03:59:42.182152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 03:59:42.182152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 03:59:42.182152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 03:59:42.182152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 03:59:42.182152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 03:59:42.182152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 03:59:42.182152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 03:59:42.182152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 03:59:42.182152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 03:59:42.183363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 03:59:42.183363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 03:59:42.183363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception Jun 24 03:59:42.183363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 03:59:42.183363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 03:59:42.183363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 03:59:42.183363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 03:59:42.183363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 03:59:42.183363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 03:59:42.183363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 03:59:42.183363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application conn.query(q) Jun 24 03:59:42.183363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 03:59:42.183363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 03:59:42.183363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 03:59:42.183363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result.read() Jun 24 03:59:42.183363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 03:59:42.183363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 03:59:42.184454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 03:59:42.184454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 03:59:42.184454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 03:59:42.184454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 03:59:42.184454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 03:59:42.184454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 03:59:42.184454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 03:59:42.184454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 03:59:42.184454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [parameters: {'id': '34e83c648a56401b80ca563bfa371211', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 03:59:42.184454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 03:59:42.184454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:42.184454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:42.184454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:42.184454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:42.184454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 24 03:59:42.185422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 24 03:59:42.185422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 03:59:42.185422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 03:59:42.185422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 24 03:59:42.185422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:42.185422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:42.185422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:42.185422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:42.185422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:42.185422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:42.185422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:42.185422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:42.185422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:42.185422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:42.185422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:42.185422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:42.185422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:42.194601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:42.194601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 24 03:59:42.194601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 24 03:59:42.194601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:42.194601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:42.194601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 24 03:59:42.194601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 24 03:59:42.194601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 34e83c648a56401b80ca563bfa371211. Jun 24 03:59:42.194601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:42.194601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 458/927] 10.222.0.16 () {60 vars in 1140 bytes} [Fri Jun 24 03:59:41 2022] POST /identity/v3/projects => generated 262 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:42.195186 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f7cdc-65c2-4aa9-bb62-0e73d6e97e6a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:42.225001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ceb05-57a5-49f5-b5c6-417e36752dc5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:42.225001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ceb05-57a5-49f5-b5c6-417e36752dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:42.225577 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ceb05-57a5-49f5-b5c6-417e36752dc5 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:42.233878 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 459/928] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 03:59:42.236955 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599aed2-e9f2-488f-be1f-10a0feefe011 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:42.237373 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599aed2-e9f2-488f-be1f-10a0feefe011 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:42.237738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599aed2-e9f2-488f-be1f-10a0feefe011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:42.259434 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0599aed2-e9f2-488f-be1f-10a0feefe011 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:42.260894 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0599aed2-e9f2-488f-be1f-10a0feefe011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:42.304870 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f7cdc-65c2-4aa9-bb62-0e73d6e97e6a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:42.319246 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:42.339668 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 460/929] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 03:59:42.341645 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d3fb3-3905-42dd-b7e0-49f4d611b6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:42.346485 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d3fb3-3905-42dd-b7e0-49f4d611b6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:42.346485 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d3fb3-3905-42dd-b7e0-49f4d611b6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:42.346485 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d3fb3-3905-42dd-b7e0-49f4d611b6c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:42.353462 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d3fb3-3905-42dd-b7e0-49f4d611b6c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:42.355129 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3d3fb3-3905-42dd-b7e0-49f4d611b6c2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:42.357961 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:42.364671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599aed2-e9f2-488f-be1f-10a0feefe011 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:42.374748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-4c3d3fb3-3905-42dd-b7e0-49f4d611b6c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:42.374748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:42.374748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:42.374748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:42.374748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:42.374748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:42.374748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:42.374748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:42.374748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:42.374748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:42.374748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:42.374748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:42.374748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:42.374748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:42.374748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:42.375646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:42.375646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:42.375646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:42.375646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:42.375646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:42.375646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:42.375646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:42.375646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:42.375646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:42.375646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:42.375646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:42.375646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:42.375646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:42.375646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:42.375646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:42.375646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:42.375646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:42.377461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:42.377461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:42.377461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:42.377461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:42.377461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:42.377461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:42.377461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:42.377461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:42.379929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 470/930] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:42.385664 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d3fb3-3905-42dd-b7e0-49f4d611b6c2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:42.477798 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599aed2-e9f2-488f-be1f-10a0feefe011 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:42.491555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d3fb3-3905-42dd-b7e0-49f4d611b6c2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:42.522445 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e7e9b-8cc6-469e-a0e6-5ac9309e739f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:42.524165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e7e9b-8cc6-469e-a0e6-5ac9309e739f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:42.524447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e7e9b-8cc6-469e-a0e6-5ac9309e739f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:42.524725 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932e7e9b-8cc6-469e-a0e6-5ac9309e739f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:42.533240 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc5b5a-38cd-4a66-996c-a4cf4110b191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:42.533972 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc5b5a-38cd-4a66-996c-a4cf4110b191 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:42.534262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc5b5a-38cd-4a66-996c-a4cf4110b191 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:42.534573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc5b5a-38cd-4a66-996c-a4cf4110b191 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:42.535347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc5b5a-38cd-4a66-996c-a4cf4110b191 None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:42.535658 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc5b5a-38cd-4a66-996c-a4cf4110b191 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:42.537437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc5b5a-38cd-4a66-996c-a4cf4110b191 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:42.540334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932e7e9b-8cc6-469e-a0e6-5ac9309e739f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:42.550049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-932e7e9b-8cc6-469e-a0e6-5ac9309e739f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 03:59:42.550049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:42.550049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:42.550049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:42.550049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:42.550049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:42.550049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:42.550049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:42.550049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:42.550049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:42.550049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:42.550049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:42.550049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:42.550049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:42.550049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:42.550049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:42.551038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:42.551038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:42.551038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:42.551038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:42.551038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:42.551038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:42.551038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:42.551038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:42.551038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:42.551038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:42.551038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:42.551038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:42.551038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:42.551038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:42.551038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:42.551038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:42.551921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:42.551921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:42.551921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:42.551921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:42.551921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:42.551921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:42.551921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:42.551921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:42.551921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:42.551921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:42.551921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 03:59:42.551921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:42.555264 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 461/931] 10.222.0.16 () {56 vars in 1120 bytes} [Fri Jun 24 03:59:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:42.560536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 471/932] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jun 24 03:59:42 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:42.560753 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e7e9b-8cc6-469e-a0e6-5ac9309e739f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:42.616351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:42.675024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848f559-080c-4378-8b35-4de56721f2e6 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:42.676099 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848f559-080c-4378-8b35-4de56721f2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:42.676906 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848f559-080c-4378-8b35-4de56721f2e6 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:42.679329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 472/933] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:42.684333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee541098-e1e7-4210-bb93-8974d7f0b513 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:42.684628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee541098-e1e7-4210-bb93-8974d7f0b513 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:42.684917 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee541098-e1e7-4210-bb93-8974d7f0b513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:42.686286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932e7e9b-8cc6-469e-a0e6-5ac9309e739f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:42.702796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ee541098-e1e7-4210-bb93-8974d7f0b513 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:42.704570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ee541098-e1e7-4210-bb93-8974d7f0b513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:42.734132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67efbcd-2bc0-4b84-baaa-08741974d76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:42.734845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67efbcd-2bc0-4b84-baaa-08741974d76a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:42.735136 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67efbcd-2bc0-4b84-baaa-08741974d76a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:42.735594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67efbcd-2bc0-4b84-baaa-08741974d76a None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:42.736484 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67efbcd-2bc0-4b84-baaa-08741974d76a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:42.736804 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67efbcd-2bc0-4b84-baaa-08741974d76a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:42.738328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67efbcd-2bc0-4b84-baaa-08741974d76a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:42.749335 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 462/934] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Jun 24 03:59:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:42.758068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67efbcd-2bc0-4b84-baaa-08741974d76a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:42.784877 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:42.798663 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 473/935] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 03:59:42.820924 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee541098-e1e7-4210-bb93-8974d7f0b513 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:42.879141 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67efbcd-2bc0-4b84-baaa-08741974d76a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:42.886981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:42.917258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a63ce7-d560-4996-a1c0-abe9ed52fba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:42.918076 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a63ce7-d560-4996-a1c0-abe9ed52fba6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:42.918337 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a63ce7-d560-4996-a1c0-abe9ed52fba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:42.918648 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a63ce7-d560-4996-a1c0-abe9ed52fba6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:42.933361 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a63ce7-d560-4996-a1c0-abe9ed52fba6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:42.935071 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a63ce7-d560-4996-a1c0-abe9ed52fba6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:42.946304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee541098-e1e7-4210-bb93-8974d7f0b513 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:42.955937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-65a63ce7-d560-4996-a1c0-abe9ed52fba6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:42.955937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:42.955937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:42.955937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:42.955937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:42.955937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:42.955937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:42.955937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:42.955937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:42.955937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:42.955937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:42.955937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:42.955937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:42.955937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:42.955937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:42.956875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:42.956875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:42.956875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:42.956875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:42.956875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:42.956875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:42.956875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:42.956875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:42.956875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:42.956875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:42.956875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:42.956875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:42.956875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:42.956875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:42.956875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:42.956875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:42.956875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:42.957765 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:42.957765 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:42.957765 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:42.957765 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:42.960071 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 463/936] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:42.966820 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a63ce7-d560-4996-a1c0-abe9ed52fba6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:42.997748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638bc67-636a-4c50-a61d-5c80e32a396f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:42.999212 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638bc67-636a-4c50-a61d-5c80e32a396f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:43.000095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638bc67-636a-4c50-a61d-5c80e32a396f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:43.000910 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638bc67-636a-4c50-a61d-5c80e32a396f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:43.002052 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c638bc67-636a-4c50-a61d-5c80e32a396f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:43.004392 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c638bc67-636a-4c50-a61d-5c80e32a396f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:43.021177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-c638bc67-636a-4c50-a61d-5c80e32a396f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 03:59:43.021177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:43.021177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:43.021177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:43.021177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:43.021177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:43.021177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:43.021177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:43.021177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 03:59:43.021177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:43.021177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:43.021177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:43.021177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:43.021177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:43.021177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:43.021177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:43.021177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:43.024152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:43.024152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:43.024152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:43.024152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:43.024152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:43.024152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:43.024152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:43.024152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:43.024152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:43.024152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:43.024152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:43.024152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:43.024152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:43.024152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:43.024152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:43.024152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:43.025007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:43.025007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:43.025007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:43.025007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:43.025007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:43.025007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:43.025007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:43.025007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:43.025007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:43.025007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:43.025007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:43.025007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:43.025007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 03:59:43.025007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:43.025007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 474/937] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:43.028765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638bc67-636a-4c50-a61d-5c80e32a396f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:43.086885 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a63ce7-d560-4996-a1c0-abe9ed52fba6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:43.111911 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:43.131490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba4ace-7502-4e5a-a8fc-c29038474057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:43.132564 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba4ace-7502-4e5a-a8fc-c29038474057 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:43.133058 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba4ace-7502-4e5a-a8fc-c29038474057 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:43.133539 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfba4ace-7502-4e5a-a8fc-c29038474057 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:43.134527 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba4ace-7502-4e5a-a8fc-c29038474057 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:43.135086 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba4ace-7502-4e5a-a8fc-c29038474057 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:43.136688 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfba4ace-7502-4e5a-a8fc-c29038474057 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:43.154401 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 464/938] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:43.162083 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba4ace-7502-4e5a-a8fc-c29038474057 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:43.181624 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638bc67-636a-4c50-a61d-5c80e32a396f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:43.185726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:43.227788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ca8300-c314-4c40-8ffa-45d41d2451ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:43.229138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca8300-c314-4c40-8ffa-45d41d2451ef None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:43.229825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca8300-c314-4c40-8ffa-45d41d2451ef None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:43.230278 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ca8300-c314-4c40-8ffa-45d41d2451ef None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:43.231840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca8300-c314-4c40-8ffa-45d41d2451ef None admin] RBAC: Adding query filter params (name=compute) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:43.232312 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca8300-c314-4c40-8ffa-45d41d2451ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:43.234344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ca8300-c314-4c40-8ffa-45d41d2451ef None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:43.241116 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 475/939] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jun 24 03:59:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:43.242671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8818e7-7def-4255-8e19-69f6b78aef3f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:43.243245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8818e7-7def-4255-8e19-69f6b78aef3f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:43.243671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8818e7-7def-4255-8e19-69f6b78aef3f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:43.245653 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 476/940] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:43.246594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8818e7-7def-4255-8e19-69f6b78aef3f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:43.277142 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfba4ace-7502-4e5a-a8fc-c29038474057 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:43.312924 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2c3a3-2084-4aa6-a7e8-71d56dc85c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:43.314289 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2c3a3-2084-4aa6-a7e8-71d56dc85c71 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:43.314601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2c3a3-2084-4aa6-a7e8-71d56dc85c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:43.314883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2c3a3-2084-4aa6-a7e8-71d56dc85c71 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:43.322436 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2c3a3-2084-4aa6-a7e8-71d56dc85c71 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:43.324559 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2c3a3-2084-4aa6-a7e8-71d56dc85c71 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:43.339840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-76f2c3a3-2084-4aa6-a7e8-71d56dc85c71 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 03:59:43.339840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:43.339840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:43.339840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:43.339840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:43.339840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:43.339840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:43.339840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:43.339840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:43.339840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:43.339840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:43.339840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:43.339840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:43.339840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:43.339840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:43.339840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:43.340826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:43.340826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:43.340826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:43.340826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:43.340826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:43.340826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:43.340826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:43.340826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:43.340826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:43.340826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:43.340826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:43.340826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:43.340826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:43.340826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:43.340826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:43.340826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:43.340826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:43.340826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:43.342794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:43.342794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:43.342794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:43.342794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:43.342794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:43.342794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:43.342794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:43.342794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:43.342794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 03:59:43.342794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:43.346221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 465/941] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jun 24 03:59:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:43.347379 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1247feb-5035-4178-b612-158d20024068 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:43.347622 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1247feb-5035-4178-b612-158d20024068 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:43.347903 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1247feb-5035-4178-b612-158d20024068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:43.355973 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8818e7-7def-4255-8e19-69f6b78aef3f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:43.365714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f1247feb-5035-4178-b612-158d20024068 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:43.366936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f1247feb-5035-4178-b612-158d20024068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:43.375499 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:43.394249 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fb3fd-fcaf-49f9-941c-6a6bc9ce3eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:43.394878 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fb3fd-fcaf-49f9-941c-6a6bc9ce3eaf None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:43.395161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fb3fd-fcaf-49f9-941c-6a6bc9ce3eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:43.395441 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fb3fd-fcaf-49f9-941c-6a6bc9ce3eaf None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:43.396156 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0fb3fd-fcaf-49f9-941c-6a6bc9ce3eaf None admin] RBAC: Adding query filter params (type=compute) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:43.396462 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0fb3fd-fcaf-49f9-941c-6a6bc9ce3eaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:43.397840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0fb3fd-fcaf-49f9-941c-6a6bc9ce3eaf None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:43.402888 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 477/942] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jun 24 03:59:43 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:43.403731 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fb3fd-fcaf-49f9-941c-6a6bc9ce3eaf None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:43.464541 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 466/943] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:43.489153 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1247feb-5035-4178-b612-158d20024068 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:43.502408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:43.530340 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fb3fd-fcaf-49f9-941c-6a6bc9ce3eaf None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:43.571778 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44783438-123e-4594-b54c-1b0353b080b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:43.572611 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44783438-123e-4594-b54c-1b0353b080b4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:43.572899 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44783438-123e-4594-b54c-1b0353b080b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:43.573178 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44783438-123e-4594-b54c-1b0353b080b4 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:43.574026 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44783438-123e-4594-b54c-1b0353b080b4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:43.574494 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44783438-123e-4594-b54c-1b0353b080b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:43.575952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44783438-123e-4594-b54c-1b0353b080b4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:43.597604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 478/944] 10.222.0.16 () {56 vars in 1149 bytes} [Fri Jun 24 03:59:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:43.603090 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44783438-123e-4594-b54c-1b0353b080b4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:43.629927 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1247feb-5035-4178-b612-158d20024068 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:43.646564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:43.668454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e99e87-8d8d-4cdc-9f7c-fffa77e3b2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:43.669652 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e99e87-8d8d-4cdc-9f7c-fffa77e3b2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:43.670157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e99e87-8d8d-4cdc-9f7c-fffa77e3b2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:43.670667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e99e87-8d8d-4cdc-9f7c-fffa77e3b2c1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:43.671517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e99e87-8d8d-4cdc-9f7c-fffa77e3b2c1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:43.673251 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e99e87-8d8d-4cdc-9f7c-fffa77e3b2c1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:43.687461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-17e99e87-8d8d-4cdc-9f7c-fffa77e3b2c1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 03:59:43.687461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:43.687461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:43.687461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:43.687461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:43.687461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:43.687461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:43.687461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:43.687461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 03:59:43.687461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:43.687461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:43.687461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:43.687461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:43.687461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:43.687461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:43.687461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:43.687461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:43.689537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:43.689537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:43.689537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:43.689537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:43.689537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:43.689537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:43.689537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:43.689537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:43.689537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:43.689537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:43.689537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:43.689537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:43.689537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:43.689537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:43.689537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:43.689537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:43.690462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:43.690462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:43.690462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:43.690462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:43.690462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:43.690462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:43.690462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:43.690462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:43.690462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:43.690462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:43.690462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:43.690462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:43.690462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 03:59:43.690462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:43.698031 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 467/945] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jun 24 03:59:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:43.699595 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e99e87-8d8d-4cdc-9f7c-fffa77e3b2c1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:43.705807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44783438-123e-4594-b54c-1b0353b080b4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:43.743099 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02163118-cfba-4531-be26-00e2492a80f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:43.743954 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02163118-cfba-4531-be26-00e2492a80f0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:43.744517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02163118-cfba-4531-be26-00e2492a80f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:43.744799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02163118-cfba-4531-be26-00e2492a80f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:43.756275 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02163118-cfba-4531-be26-00e2492a80f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:43.757889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02163118-cfba-4531-be26-00e2492a80f0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:43.777613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-02163118-cfba-4531-be26-00e2492a80f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:43.777613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:43.777613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:43.777613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:43.777613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:43.777613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:43.777613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:43.777613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:43.777613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:43.777613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:43.777613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:43.777613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:43.777613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:43.777613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:43.777613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:43.778620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:43.778620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:43.778620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:43.778620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:43.778620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:43.778620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:43.778620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:43.778620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:43.778620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:43.778620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:43.778620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:43.778620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:43.778620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:43.778620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:43.778620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:43.778620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:43.778620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:43.779510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:43.779510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:43.779510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:43.779510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:43.784082 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 479/946] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:43.785027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02163118-cfba-4531-be26-00e2492a80f0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:43.789861 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:43.820709 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e99e87-8d8d-4cdc-9f7c-fffa77e3b2c1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:43.863449 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2b83d-3ff8-4cac-8b3e-a2a926243a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:43.864656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2b83d-3ff8-4cac-8b3e-a2a926243a88 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:43.865148 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2b83d-3ff8-4cac-8b3e-a2a926243a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:43.865619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2b83d-3ff8-4cac-8b3e-a2a926243a88 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:43.866554 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2b83d-3ff8-4cac-8b3e-a2a926243a88 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:43.867062 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2b83d-3ff8-4cac-8b3e-a2a926243a88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:43.868642 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2b83d-3ff8-4cac-8b3e-a2a926243a88 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:43.879353 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 468/947] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jun 24 03:59:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:43.881360 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2b83d-3ff8-4cac-8b3e-a2a926243a88 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:43.900215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:43.913604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02163118-cfba-4531-be26-00e2492a80f0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:43.959341 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb0f80-71a6-47dd-af46-289529fbe955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:43.960185 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb0f80-71a6-47dd-af46-289529fbe955 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:43.960573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb0f80-71a6-47dd-af46-289529fbe955 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:43.961726 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb0f80-71a6-47dd-af46-289529fbe955 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:43.962611 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb0f80-71a6-47dd-af46-289529fbe955 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:43.963047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb0f80-71a6-47dd-af46-289529fbe955 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:43.964608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb0f80-71a6-47dd-af46-289529fbe955 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:43.982637 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 480/948] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:43.988632 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb0f80-71a6-47dd-af46-289529fbe955 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:44.000911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2b83d-3ff8-4cac-8b3e-a2a926243a88 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:44.039884 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847247f-f0d3-4ac3-b299-a33fb453bbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:44.041074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847247f-f0d3-4ac3-b299-a33fb453bbb6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:44.041074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847247f-f0d3-4ac3-b299-a33fb453bbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:44.041270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847247f-f0d3-4ac3-b299-a33fb453bbb6 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:44.042093 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847247f-f0d3-4ac3-b299-a33fb453bbb6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:44.042500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847247f-f0d3-4ac3-b299-a33fb453bbb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:44.044169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847247f-f0d3-4ac3-b299-a33fb453bbb6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:44.046708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:44.051235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 469/949] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jun 24 03:59:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:44.058754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847247f-f0d3-4ac3-b299-a33fb453bbb6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:44.113007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb0f80-71a6-47dd-af46-289529fbe955 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:44.162521 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da205706-ff99-4609-9deb-2327ad97d0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:44.164219 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da205706-ff99-4609-9deb-2327ad97d0b7 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:44.164705 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da205706-ff99-4609-9deb-2327ad97d0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:44.165165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da205706-ff99-4609-9deb-2327ad97d0b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:44.175544 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da205706-ff99-4609-9deb-2327ad97d0b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:44.177884 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da205706-ff99-4609-9deb-2327ad97d0b7 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:44.180414 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847247f-f0d3-4ac3-b299-a33fb453bbb6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:44.197834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-da205706-ff99-4609-9deb-2327ad97d0b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:44.197834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:44.197834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:44.197834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:44.197834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:44.197834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:44.197834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:44.197834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:44.197834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:44.197834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:44.197834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:44.197834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:44.197834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:44.197834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:44.197834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:44.199029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:44.199029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:44.199029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:44.199029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:44.199029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:44.199029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:44.199029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:44.199029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:44.199029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:44.199029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:44.199029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:44.199029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:44.199029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:44.199029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:44.199029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:44.199029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:44.199029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:44.199990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:44.199990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:44.199990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:44.199990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:44.199990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:44.199990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:44.199990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:44.199990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:44.200436 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 481/950] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:44.209638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da205706-ff99-4609-9deb-2327ad97d0b7 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:44.225798 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f4a38-67f4-4c6a-adb3-1bdbb2f733ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:44.226376 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f4a38-67f4-4c6a-adb3-1bdbb2f733ce None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:44.226680 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f4a38-67f4-4c6a-adb3-1bdbb2f733ce None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:44.226958 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f4a38-67f4-4c6a-adb3-1bdbb2f733ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:44.227744 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f4a38-67f4-4c6a-adb3-1bdbb2f733ce None admin] RBAC: Adding query filter params (interface=public, service_id=e7f2e107823d411391004b433f3e068f, region_id=RegionOne) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:44.228054 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f4a38-67f4-4c6a-adb3-1bdbb2f733ce None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:44.229421 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f4a38-67f4-4c6a-adb3-1bdbb2f733ce None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:44.232447 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:44.239968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 470/951] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Jun 24 03:59:44 2022] GET /identity/v3/endpoints?service_id=e7f2e107823d411391004b433f3e068f&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:44.241303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f4a38-67f4-4c6a-adb3-1bdbb2f733ce None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:44.315598 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:44.339067 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da205706-ff99-4609-9deb-2327ad97d0b7 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:44.370083 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f4a38-67f4-4c6a-adb3-1bdbb2f733ce None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:44.377962 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00651e3-201d-421f-9177-a6f968df61f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:44.378794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00651e3-201d-421f-9177-a6f968df61f4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:44.379517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00651e3-201d-421f-9177-a6f968df61f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:44.379639 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00651e3-201d-421f-9177-a6f968df61f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:44.380513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00651e3-201d-421f-9177-a6f968df61f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:44.380921 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00651e3-201d-421f-9177-a6f968df61f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:44.382568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00651e3-201d-421f-9177-a6f968df61f4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:44.400951 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 482/952] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:44.402013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00651e3-201d-421f-9177-a6f968df61f4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:44.417271 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82b251-1533-478d-911b-1d6a5c3f822c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:44.418069 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82b251-1533-478d-911b-1d6a5c3f822c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:44.418386 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82b251-1533-478d-911b-1d6a5c3f822c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:44.418775 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82b251-1533-478d-911b-1d6a5c3f822c None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:44.419562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82b251-1533-478d-911b-1d6a5c3f822c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:44.421005 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82b251-1533-478d-911b-1d6a5c3f822c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:44.434548 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 471/953] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Jun 24 03:59:44 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 03:59:44.487201 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:44.523668 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00651e3-201d-421f-9177-a6f968df61f4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:44.565504 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86f866-7410-4691-ba38-928b779bd003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:44.566713 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86f866-7410-4691-ba38-928b779bd003 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:44.567155 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86f866-7410-4691-ba38-928b779bd003 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:44.567587 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86f866-7410-4691-ba38-928b779bd003 None admin] PATH_INFO: `/v3/projects/68d376986e854a4bb334676572340b4c/users/98f58f91d17949198e66883d234a9e40/roles/254f32a34d7b4de7b5c039848b8153d6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:44.570122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86f866-7410-4691-ba38-928b779bd003 None admin] RBAC: Authorizing `identity:create_grant(project_id=68d376986e854a4bb334676572340b4c, user_id=98f58f91d17949198e66883d234a9e40, role_id=254f32a34d7b4de7b5c039848b8153d6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:44.571832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86f866-7410-4691-ba38-928b779bd003 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:44.667732 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 483/954] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 03:59:44 2022] PUT /identity/v3/projects/68d376986e854a4bb334676572340b4c/users/98f58f91d17949198e66883d234a9e40/roles/254f32a34d7b4de7b5c039848b8153d6 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 03:59:44.788104 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb621c2a-b000-4c11-be15-94a35f9da698 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:44.788397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb621c2a-b000-4c11-be15-94a35f9da698 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:44.788683 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb621c2a-b000-4c11-be15-94a35f9da698 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:44.790632 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 472/955] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:44.796245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ef9d8-5d31-4806-a88c-b8c3872a6b5d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:44.796539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ef9d8-5d31-4806-a88c-b8c3872a6b5d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:44.796809 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ef9d8-5d31-4806-a88c-b8c3872a6b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:44.816861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ca8ef9d8-5d31-4806-a88c-b8c3872a6b5d None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:44.818132 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ca8ef9d8-5d31-4806-a88c-b8c3872a6b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:44.892166 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 484/956] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:44.917289 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb621c2a-b000-4c11-be15-94a35f9da698 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:45.036435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb621c2a-b000-4c11-be15-94a35f9da698 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:45.069900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:45.077892 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdefff2-b35a-4c83-b765-075f9e012b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:45.078601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdefff2-b35a-4c83-b765-075f9e012b31 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:45.078966 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdefff2-b35a-4c83-b765-075f9e012b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:45.079300 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdefff2-b35a-4c83-b765-075f9e012b31 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:45.080213 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdefff2-b35a-4c83-b765-075f9e012b31 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:45.081646 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdefff2-b35a-4c83-b765-075f9e012b31 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:45.129076 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.sql.core [None req-6bdefff2-b35a-4c83-b765-075f9e012b31 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 03:59:45.129076 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 03:59:45.129076 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [parameters: {'user_id': '6436dd1c5b3341458eca1bbb5e90c3c6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 03:59:45.129076 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80600) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 03:59:45.137446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-6bdefff2-b35a-4c83-b765-075f9e012b31 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 03:59:45.137446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:45.137446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 03:59:45.137446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 03:59:45.137446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 03:59:45.137446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 03:59:45.137446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 03:59:45.137446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 03:59:45.137446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 03:59:45.137446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application conn.query(q) Jun 24 03:59:45.137446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 03:59:45.137446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 03:59:45.137446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 03:59:45.137446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result.read() Jun 24 03:59:45.137446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 03:59:45.142645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 03:59:45.142645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 03:59:45.142645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 03:59:45.142645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 03:59:45.142645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 03:59:45.142645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 03:59:45.142645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 03:59:45.142645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 03:59:45.142645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:45.142645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 03:59:45.142645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:45.142645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:45.142645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 03:59:45.142645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 03:59:45.142645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 24 03:59:45.142645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 24 03:59:45.142645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 03:59:45.142645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application next(self.gen) Jun 24 03:59:45.143669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 03:59:45.143669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application yield resource Jun 24 03:59:45.143669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 03:59:45.143669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application next(self.gen) Jun 24 03:59:45.143669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 03:59:45.143669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 03:59:45.143669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 03:59:45.143669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 03:59:45.143669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 03:59:45.143669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise self.value Jun 24 03:59:45.143669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 03:59:45.143669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 03:59:45.143669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 03:59:45.143669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application session.commit() Jun 24 03:59:45.143669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 03:59:45.143669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 03:59:45.143669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 03:59:45.144783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 03:59:45.144783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 03:59:45.144783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.session.flush() Jun 24 03:59:45.144783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 03:59:45.144783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 03:59:45.144783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 03:59:45.144783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 03:59:45.144783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 03:59:45.144783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application compat.raise_( Jun 24 03:59:45.144783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 03:59:45.144783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception Jun 24 03:59:45.144783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 03:59:45.144783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 03:59:45.144783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 03:59:45.144783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 03:59:45.144783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 03:59:45.144783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 03:59:45.145895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 03:59:45.145895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 03:59:45.145895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 24 03:59:45.145895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 24 03:59:45.145895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 03:59:45.145895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 03:59:45.145895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 03:59:45.145895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 03:59:45.145895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 03:59:45.145895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 03:59:45.145895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 03:59:45.145895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 03:59:45.145895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 03:59:45.145895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 03:59:45.145895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 03:59:45.145895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception Jun 24 03:59:45.147044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 03:59:45.147044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 03:59:45.147044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 03:59:45.147044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 03:59:45.147044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 03:59:45.147044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 03:59:45.147044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 03:59:45.147044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application conn.query(q) Jun 24 03:59:45.147044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 03:59:45.147044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 03:59:45.147044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 03:59:45.147044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application result.read() Jun 24 03:59:45.147044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 03:59:45.147044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 03:59:45.147044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 03:59:45.147044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 03:59:45.148030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 03:59:45.148030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 03:59:45.148030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 03:59:45.148030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 03:59:45.148030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 03:59:45.148030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 03:59:45.148030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [parameters: {'user_id': '6436dd1c5b3341458eca1bbb5e90c3c6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 03:59:45.148030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 03:59:45.148030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:45.148030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:45.148030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:45.148030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:45.148030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:45.148030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:45.148030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:45.148944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:45.148944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:45.148944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:45.148944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:45.148944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:45.148944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:45.148944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:45.148944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 24 03:59:45.148944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 24 03:59:45.148944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:45.148944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:45.148944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:45.148944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:45.148944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:45.148944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:45.148944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 24 03:59:45.148944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 24 03:59:45.149946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 24 03:59:45.149946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 24 03:59:45.149946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 03:59:45.149946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 03:59:45.149946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 03:59:45.149946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:45.149946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 473/957] 10.222.0.16 () {60 vars in 1131 bytes} [Fri Jun 24 03:59:44 2022] POST /identity/v3/users => generated 160 bytes in 222 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:45.152607 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ef9d8-5d31-4806-a88c-b8c3872a6b5d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:45.221917 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:45.265014 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ef9d8-5d31-4806-a88c-b8c3872a6b5d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:45.303674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bdfb5-1438-4d05-8246-351e3946d90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:45.304678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bdfb5-1438-4d05-8246-351e3946d90b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:45.305124 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bdfb5-1438-4d05-8246-351e3946d90b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:45.305558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bdfb5-1438-4d05-8246-351e3946d90b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:45.312993 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19bdfb5-1438-4d05-8246-351e3946d90b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:45.314644 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19bdfb5-1438-4d05-8246-351e3946d90b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:45.329421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-b19bdfb5-1438-4d05-8246-351e3946d90b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 03:59:45.329421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:45.329421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:45.329421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:45.329421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:45.329421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:45.329421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:45.329421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:45.329421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:45.329421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:45.329421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:45.329421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:45.329421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:45.329421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:45.329421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:45.329421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:45.330631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:45.330631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:45.330631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:45.330631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:45.330631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:45.330631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:45.330631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:45.330631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:45.330631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:45.330631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:45.330631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:45.330631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:45.330631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:45.330631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:45.330631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:45.330631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:45.330631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:45.330631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:45.331843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:45.331843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:45.331843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:45.331843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:45.331843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:45.331843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:45.331843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:45.331843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:45.331843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 03:59:45.331843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:45.334180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 485/958] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 03:59:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:45.340087 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdefff2-b35a-4c83-b765-075f9e012b31 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:45.456768 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdefff2-b35a-4c83-b765-075f9e012b31 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:45.493655 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c2903-e471-4015-8853-dbf52304857d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:45.494616 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c2903-e471-4015-8853-dbf52304857d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:45.494903 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c2903-e471-4015-8853-dbf52304857d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:45.495180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c2903-e471-4015-8853-dbf52304857d None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:45.495989 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c2903-e471-4015-8853-dbf52304857d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:45.496300 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c2903-e471-4015-8853-dbf52304857d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:45.497952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c2903-e471-4015-8853-dbf52304857d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:45.512489 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15cd70-2e60-45a1-a015-a1d82314b613 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:45.512785 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15cd70-2e60-45a1-a015-a1d82314b613 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:45.513074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15cd70-2e60-45a1-a015-a1d82314b613 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:45.514719 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 486/959] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 03:59:45.520074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9755b-37e1-44b3-b44f-f0f16b03829d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:45.520370 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9755b-37e1-44b3-b44f-f0f16b03829d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:45.520655 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9755b-37e1-44b3-b44f-f0f16b03829d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:45.538233 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0ca9755b-37e1-44b3-b44f-f0f16b03829d None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:45.539473 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0ca9755b-37e1-44b3-b44f-f0f16b03829d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:45.541009 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 474/960] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Jun 24 03:59:45 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:45.623195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 487/961] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 03:59:45.647724 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c2903-e471-4015-8853-dbf52304857d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:45.712272 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:45.772428 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c2903-e471-4015-8853-dbf52304857d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:45.810815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d49d0-0fc0-4629-8051-8ac632241a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:45.811601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d49d0-0fc0-4629-8051-8ac632241a3e None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:45.812020 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d49d0-0fc0-4629-8051-8ac632241a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:45.812423 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d49d0-0fc0-4629-8051-8ac632241a3e None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:45.813169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d49d0-0fc0-4629-8051-8ac632241a3e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:45.815686 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d49d0-0fc0-4629-8051-8ac632241a3e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:45.914140 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 475/962] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jun 24 03:59:45 2022] POST /identity/v3/services => generated 242 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:59:46.632686 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a47ec-92d2-4df0-b890-ad661dc2bfca None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:46.633209 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a47ec-92d2-4df0-b890-ad661dc2bfca None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:46.633289 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a47ec-92d2-4df0-b890-ad661dc2bfca None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:46.634685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 488/963] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:46.639875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83e7b9-92f1-4bec-84d9-948fa0045af6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:46.640258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83e7b9-92f1-4bec-84d9-948fa0045af6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:46.640447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83e7b9-92f1-4bec-84d9-948fa0045af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:46.657762 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8f83e7b9-92f1-4bec-84d9-948fa0045af6 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:46.659333 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8f83e7b9-92f1-4bec-84d9-948fa0045af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:46.733230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 476/964] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:46.758725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a47ec-92d2-4df0-b890-ad661dc2bfca None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:46.813277 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:46.821899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce1e7a-2fb0-40a7-a852-dc78f8b7e767 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:46.822232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce1e7a-2fb0-40a7-a852-dc78f8b7e767 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:46.822568 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce1e7a-2fb0-40a7-a852-dc78f8b7e767 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:46.826870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 477/965] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:46.828754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2dd33-3899-49f0-bebc-b2b5c53c4da7 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:46.829232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2dd33-3899-49f0-bebc-b2b5c53c4da7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:46.829549 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2dd33-3899-49f0-bebc-b2b5c53c4da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:46.846993 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b7f2dd33-3899-49f0-bebc-b2b5c53c4da7 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:46.848218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b7f2dd33-3899-49f0-bebc-b2b5c53c4da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:46.866456 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a47ec-92d2-4df0-b890-ad661dc2bfca None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:46.877816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:46.902649 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0d3be-e858-47ad-9935-ab616fb997ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:46.903385 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0d3be-e858-47ad-9935-ab616fb997ab None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:46.903667 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0d3be-e858-47ad-9935-ab616fb997ab None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:46.903953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0d3be-e858-47ad-9935-ab616fb997ab None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:46.904615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0d3be-e858-47ad-9935-ab616fb997ab None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:46.905994 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0d3be-e858-47ad-9935-ab616fb997ab None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:46.919414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-89f0d3be-e858-47ad-9935-ab616fb997ab None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 03:59:46.919414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:46.919414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:46.919414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:46.919414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:46.919414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:46.919414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:46.919414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:46.919414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 03:59:46.919414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:46.919414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:46.919414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:46.919414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:46.919414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:46.919414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:46.919414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:46.919414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:46.920483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:46.920483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:46.920483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:46.920483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:46.920483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:46.920483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:46.920483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:46.920483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:46.920483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:46.920483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:46.920483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:46.920483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:46.920483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:46.920483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:46.920483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:46.920483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:46.921492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:46.921492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:46.921492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:46.921492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:46.921492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:46.921492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:46.921492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:46.921492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:46.921492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:46.921492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:46.921492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:46.921492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:46.921492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 03:59:46.921492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:46.921492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 489/966] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jun 24 03:59:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:46.926045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 478/967] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 03:59:46.927461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0d3be-e858-47ad-9935-ab616fb997ab None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:46.946670 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2dd33-3899-49f0-bebc-b2b5c53c4da7 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:47.046580 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0d3be-e858-47ad-9935-ab616fb997ab None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:47.063122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:47.065236 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2dd33-3899-49f0-bebc-b2b5c53c4da7 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:47.083845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb1ca2-9a42-47eb-ad0a-1a8563ec0883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:47.084509 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb1ca2-9a42-47eb-ad0a-1a8563ec0883 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:47.084793 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb1ca2-9a42-47eb-ad0a-1a8563ec0883 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:47.085103 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb1ca2-9a42-47eb-ad0a-1a8563ec0883 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:47.085891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb1ca2-9a42-47eb-ad0a-1a8563ec0883 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:47.086196 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb1ca2-9a42-47eb-ad0a-1a8563ec0883 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:47.087596 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb1ca2-9a42-47eb-ad0a-1a8563ec0883 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:47.093500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 490/968] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jun 24 03:59:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:47.099248 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb1ca2-9a42-47eb-ad0a-1a8563ec0883 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:47.106744 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08bf20-9604-4bc2-968d-400e71dae48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:47.107576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08bf20-9604-4bc2-968d-400e71dae48c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:47.107899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08bf20-9604-4bc2-968d-400e71dae48c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:47.108224 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b08bf20-9604-4bc2-968d-400e71dae48c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:47.119818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b08bf20-9604-4bc2-968d-400e71dae48c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:47.128235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-5b08bf20-9604-4bc2-968d-400e71dae48c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 03:59:47.128235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:47.128235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:47.128235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:47.128235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:47.128235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:47.128235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:47.128235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:47.128235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:47.128235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:47.128235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:47.128235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:47.128235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:47.128235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:47.128235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:47.128235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:47.129368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:47.129368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:47.129368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:47.129368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:47.129368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:47.129368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:47.129368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:47.129368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:47.129368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:47.129368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:47.129368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:47.129368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:47.129368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:47.129368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:47.129368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:47.129368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:47.130326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:47.130326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:47.130326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:47.130326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:47.130326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:47.130326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:47.130326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:47.130326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:47.130326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:47.130326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:47.130326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 03:59:47.130326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:47.130326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 479/969] 10.222.0.16 () {56 vars in 1120 bytes} [Fri Jun 24 03:59:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:47.135238 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08bf20-9604-4bc2-968d-400e71dae48c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:47.164292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:47.210055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb1ca2-9a42-47eb-ad0a-1a8563ec0883 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:47.247027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b3e62-7e5e-4d15-8e33-9ac9880a09b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:47.247442 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b08bf20-9604-4bc2-968d-400e71dae48c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:47.248161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b3e62-7e5e-4d15-8e33-9ac9880a09b3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:47.248622 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b3e62-7e5e-4d15-8e33-9ac9880a09b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:47.249072 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b3e62-7e5e-4d15-8e33-9ac9880a09b3 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:47.249959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b3e62-7e5e-4d15-8e33-9ac9880a09b3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:47.250266 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b3e62-7e5e-4d15-8e33-9ac9880a09b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:47.252008 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b3e62-7e5e-4d15-8e33-9ac9880a09b3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:47.257289 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 491/970] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jun 24 03:59:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:47.262441 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b3e62-7e5e-4d15-8e33-9ac9880a09b3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:47.288354 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd1cd0-762e-43a2-86a5-4cea00a9eecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:47.289111 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd1cd0-762e-43a2-86a5-4cea00a9eecd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:47.289397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd1cd0-762e-43a2-86a5-4cea00a9eecd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:47.289675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd1cd0-762e-43a2-86a5-4cea00a9eecd None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:47.290491 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd1cd0-762e-43a2-86a5-4cea00a9eecd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:47.290793 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd1cd0-762e-43a2-86a5-4cea00a9eecd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:47.292204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd1cd0-762e-43a2-86a5-4cea00a9eecd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:47.300594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 480/971] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Jun 24 03:59:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:47.306061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd1cd0-762e-43a2-86a5-4cea00a9eecd None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:47.311275 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:47.373354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b3e62-7e5e-4d15-8e33-9ac9880a09b3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:47.396653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:47.412165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd1cd0-762e-43a2-86a5-4cea00a9eecd None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:47.412318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fa147-a1ef-45f6-b6f7-de2efc6e12a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:47.412972 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fa147-a1ef-45f6-b6f7-de2efc6e12a8 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:47.413263 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fa147-a1ef-45f6-b6f7-de2efc6e12a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:47.413624 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fa147-a1ef-45f6-b6f7-de2efc6e12a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:47.415055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fa147-a1ef-45f6-b6f7-de2efc6e12a8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:47.415693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fa147-a1ef-45f6-b6f7-de2efc6e12a8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:47.448367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522620f-9747-48e2-b302-c3b05c6464d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:47.449157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522620f-9747-48e2-b302-c3b05c6464d9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:47.449441 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522620f-9747-48e2-b302-c3b05c6464d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:47.449720 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522620f-9747-48e2-b302-c3b05c6464d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:47.461023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522620f-9747-48e2-b302-c3b05c6464d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:47.462670 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522620f-9747-48e2-b302-c3b05c6464d9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:47.479772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-0522620f-9747-48e2-b302-c3b05c6464d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:47.479772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:47.479772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:47.479772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:47.479772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:47.479772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:47.479772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:47.479772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:47.479772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:47.479772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:47.479772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:47.479772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:47.479772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:47.479772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:47.479772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:47.480714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:47.480714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:47.480714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:47.480714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:47.480714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:47.480714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:47.480714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:47.480714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:47.480714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:47.480714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:47.480714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:47.480714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:47.480714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:47.480714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:47.480714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:47.480714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:47.480714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:47.481747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:47.481747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:47.481747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:47.481747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:47.481747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 481/972] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:47.487025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522620f-9747-48e2-b302-c3b05c6464d9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:47.506400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 492/973] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Jun 24 03:59:47 2022] POST /identity/v3/endpoints => generated 349 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 03:59:47.611860 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522620f-9747-48e2-b302-c3b05c6464d9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:47.655717 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48a724-b278-412b-94fa-9954d2d67c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:47.656351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48a724-b278-412b-94fa-9954d2d67c97 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:47.656351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48a724-b278-412b-94fa-9954d2d67c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:47.656671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48a724-b278-412b-94fa-9954d2d67c97 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:47.657552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48a724-b278-412b-94fa-9954d2d67c97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:47.657870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48a724-b278-412b-94fa-9954d2d67c97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:47.659867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48a724-b278-412b-94fa-9954d2d67c97 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:47.661215 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:47.673003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 482/974] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:47.678783 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fa147-a1ef-45f6-b6f7-de2efc6e12a8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:47.788256 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:47.792136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fa147-a1ef-45f6-b6f7-de2efc6e12a8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:47.832542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ff657-6ae1-4e1f-9e35-4f8775969f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:47.833864 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ff657-6ae1-4e1f-9e35-4f8775969f98 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:47.834347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ff657-6ae1-4e1f-9e35-4f8775969f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:47.835022 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ff657-6ae1-4e1f-9e35-4f8775969f98 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:47.845640 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247ff657-6ae1-4e1f-9e35-4f8775969f98 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:47.848010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247ff657-6ae1-4e1f-9e35-4f8775969f98 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:47.865807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-247ff657-6ae1-4e1f-9e35-4f8775969f98 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 03:59:47.865807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:47.865807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:47.865807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:47.865807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:47.865807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:47.865807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:47.865807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:47.865807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:47.865807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:47.865807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:47.865807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:47.865807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:47.865807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:47.865807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:47.865807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:47.866891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:47.866891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:47.866891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:47.866891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:47.866891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:47.866891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:47.866891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:47.866891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:47.866891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:47.866891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:47.866891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:47.866891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:47.866891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:47.866891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:47.866891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:47.866891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:47.866891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:47.866891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:47.872763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:47.872763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:47.872763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:47.872763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:47.872763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:47.872763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:47.872763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:47.872763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:47.872763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 03:59:47.872763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:47.872763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 493/975] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jun 24 03:59:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:47.874802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48a724-b278-412b-94fa-9954d2d67c97 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:47.979318 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48a724-b278-412b-94fa-9954d2d67c97 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:47.983393 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1af0df-546c-472b-a495-4d861258d8bf None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:47.983688 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1af0df-546c-472b-a495-4d861258d8bf None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:47.983971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1af0df-546c-472b-a495-4d861258d8bf None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:47.986412 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 494/976] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 03:59:47.991129 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765546a-0161-4c06-bf17-979222c2294c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:47.991638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765546a-0161-4c06-bf17-979222c2294c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:47.991811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765546a-0161-4c06-bf17-979222c2294c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:48.011286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0765546a-0161-4c06-bf17-979222c2294c None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:48.012460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0765546a-0161-4c06-bf17-979222c2294c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:48.018385 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ade8b-192b-4776-b339-0f67dbc65fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:48.019062 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ade8b-192b-4776-b339-0f67dbc65fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:48.019349 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ade8b-192b-4776-b339-0f67dbc65fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:48.019628 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ade8b-192b-4776-b339-0f67dbc65fb8 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:48.020476 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ade8b-192b-4776-b339-0f67dbc65fb8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:48.020792 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ade8b-192b-4776-b339-0f67dbc65fb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:48.022220 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ade8b-192b-4776-b339-0f67dbc65fb8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:48.041096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 483/977] 10.222.0.16 () {56 vars in 1149 bytes} [Fri Jun 24 03:59:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:48.046905 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ade8b-192b-4776-b339-0f67dbc65fb8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:48.100522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 495/978] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 03:59:48.107773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:48.124386 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765546a-0161-4c06-bf17-979222c2294c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:48.171035 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ade8b-192b-4776-b339-0f67dbc65fb8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:48.211717 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c690d-9dc4-4a41-af24-719f153b7492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:48.212366 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c690d-9dc4-4a41-af24-719f153b7492 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:48.212652 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c690d-9dc4-4a41-af24-719f153b7492 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:48.212934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c690d-9dc4-4a41-af24-719f153b7492 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:48.227125 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:48.239994 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765546a-0161-4c06-bf17-979222c2294c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:48.273743 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f18be1-ecc7-403c-8afa-cfcb1e54e6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:48.274572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f18be1-ecc7-403c-8afa-cfcb1e54e6d5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:48.274871 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f18be1-ecc7-403c-8afa-cfcb1e54e6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:48.275141 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f18be1-ecc7-403c-8afa-cfcb1e54e6d5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:48.275797 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f18be1-ecc7-403c-8afa-cfcb1e54e6d5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:48.277173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f18be1-ecc7-403c-8afa-cfcb1e54e6d5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:48.285098 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c690d-9dc4-4a41-af24-719f153b7492 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:48.286575 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c690d-9dc4-4a41-af24-719f153b7492 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:48.288935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-a6f18be1-ecc7-403c-8afa-cfcb1e54e6d5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 03:59:48.288935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:48.288935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:48.288935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:48.288935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:48.288935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:48.288935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:48.288935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:48.288935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 03:59:48.288935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:48.288935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:48.288935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:48.288935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:48.288935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:48.288935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:48.288935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:48.288935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:48.289941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:48.289941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:48.289941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:48.289941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:48.289941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:48.289941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:48.289941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:48.289941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:48.289941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:48.289941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:48.289941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:48.289941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:48.289941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:48.289941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:48.289941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:48.289941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:48.290952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:48.290952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:48.290952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:48.290952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:48.290952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:48.290952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:48.290952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:48.290952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:48.290952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:48.290952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:48.290952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:48.290952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:48.290952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 03:59:48.290952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:48.290952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 496/979] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:48.295822 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f18be1-ecc7-403c-8afa-cfcb1e54e6d5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:48.343259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:48.398402 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 484/980] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:59:48 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 03:59:48.399431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c690d-9dc4-4a41-af24-719f153b7492 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:48.404641 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f18be1-ecc7-403c-8afa-cfcb1e54e6d5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:48.440358 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1294246-b3cd-481f-9126-ea3a94cdaaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:48.441023 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1294246-b3cd-481f-9126-ea3a94cdaaf9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:48.441307 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1294246-b3cd-481f-9126-ea3a94cdaaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:48.441639 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1294246-b3cd-481f-9126-ea3a94cdaaf9 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:48.442291 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1294246-b3cd-481f-9126-ea3a94cdaaf9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:48.442630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1294246-b3cd-481f-9126-ea3a94cdaaf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:48.444007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1294246-b3cd-481f-9126-ea3a94cdaaf9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:48.455121 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 497/981] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jun 24 03:59:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:48.457101 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1294246-b3cd-481f-9126-ea3a94cdaaf9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:48.476457 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:48.512370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c690d-9dc4-4a41-af24-719f153b7492 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:48.551985 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af335054-bea0-485a-892a-0a62fbec04bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:48.552781 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af335054-bea0-485a-892a-0a62fbec04bd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:48.553095 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af335054-bea0-485a-892a-0a62fbec04bd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:48.553334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af335054-bea0-485a-892a-0a62fbec04bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:48.563051 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1294246-b3cd-481f-9126-ea3a94cdaaf9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:48.564819 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af335054-bea0-485a-892a-0a62fbec04bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:48.566274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af335054-bea0-485a-892a-0a62fbec04bd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:48.584694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-af335054-bea0-485a-892a-0a62fbec04bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:48.584694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:48.584694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:48.584694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:48.584694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:48.584694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:48.584694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:48.584694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:48.584694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:48.584694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:48.584694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:48.584694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:48.584694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:48.584694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:48.584694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:48.585714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:48.585714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:48.585714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:48.585714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:48.585714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:48.585714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:48.585714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:48.585714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:48.585714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:48.585714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:48.585714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:48.585714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:48.585714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:48.585714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:48.585714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:48.585714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:48.585714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:48.586703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:48.586703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:48.586703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:48.586703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:48.586703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 485/982] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:48.591420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af335054-bea0-485a-892a-0a62fbec04bd None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:48.595546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:48.603491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa1c71-3275-46dd-b788-a788e9b2f9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:48.604163 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa1c71-3275-46dd-b788-a788e9b2f9ba None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:48.604457 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa1c71-3275-46dd-b788-a788e9b2f9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:48.605401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa1c71-3275-46dd-b788-a788e9b2f9ba None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:48.605664 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa1c71-3275-46dd-b788-a788e9b2f9ba None admin] RBAC: Adding query filter params (type=compute) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:48.605749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa1c71-3275-46dd-b788-a788e9b2f9ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:48.607127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa1c71-3275-46dd-b788-a788e9b2f9ba None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:48.613670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 498/983] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jun 24 03:59:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:48.619522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa1c71-3275-46dd-b788-a788e9b2f9ba None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:48.700472 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af335054-bea0-485a-892a-0a62fbec04bd None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:48.708806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:48.726613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa1c71-3275-46dd-b788-a788e9b2f9ba None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:48.738268 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76290c75-45f0-4dcf-a2eb-350a43240c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:48.739753 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76290c75-45f0-4dcf-a2eb-350a43240c25 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:48.740182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76290c75-45f0-4dcf-a2eb-350a43240c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:48.740457 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76290c75-45f0-4dcf-a2eb-350a43240c25 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:48.741186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76290c75-45f0-4dcf-a2eb-350a43240c25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:48.741493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76290c75-45f0-4dcf-a2eb-350a43240c25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:48.742898 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76290c75-45f0-4dcf-a2eb-350a43240c25 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:48.761131 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 486/984] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:48.762472 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76290c75-45f0-4dcf-a2eb-350a43240c25 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:48.775349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b749a01a-c797-4bc9-a7ee-755bb422efa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:48.776221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749a01a-c797-4bc9-a7ee-755bb422efa8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:48.776659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749a01a-c797-4bc9-a7ee-755bb422efa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:48.777085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b749a01a-c797-4bc9-a7ee-755bb422efa8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:48.778134 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b749a01a-c797-4bc9-a7ee-755bb422efa8 None admin] RBAC: Adding query filter params (interface=public, service_id=bffe8314e2994b93b3b8a2d3685a3bee, region_id=RegionOne) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:48.778650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b749a01a-c797-4bc9-a7ee-755bb422efa8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:48.780179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b749a01a-c797-4bc9-a7ee-755bb422efa8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:48.787888 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 499/985] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Jun 24 03:59:48 2022] GET /identity/v3/endpoints?service_id=bffe8314e2994b93b3b8a2d3685a3bee&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:48.793228 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b749a01a-c797-4bc9-a7ee-755bb422efa8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:48.831998 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:48.870005 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76290c75-45f0-4dcf-a2eb-350a43240c25 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:48.903939 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b749a01a-c797-4bc9-a7ee-755bb422efa8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:48.908070 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf1182-d313-4174-af58-31f592b153c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:48.908859 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf1182-d313-4174-af58-31f592b153c6 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:48.909234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf1182-d313-4174-af58-31f592b153c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:48.909518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf1182-d313-4174-af58-31f592b153c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:48.917433 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf1182-d313-4174-af58-31f592b153c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:48.918965 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf1182-d313-4174-af58-31f592b153c6 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:48.933521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-87cf1182-d313-4174-af58-31f592b153c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:48.933521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:48.933521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:48.933521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:48.933521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:48.933521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:48.933521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:48.933521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:48.933521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:48.933521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:48.933521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:48.933521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:48.933521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:48.933521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:48.933521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:48.934500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:48.934500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:48.934500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:48.934500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:48.934500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:48.934500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:48.934500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:48.934500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:48.934500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:48.934500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:48.934500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:48.934500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:48.934500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:48.934500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:48.934500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:48.934500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:48.934500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:48.939550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:48.939550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:48.939550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:48.939550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:48.939550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:48.939550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:48.939550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:48.939550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:48.939550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 487/986] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:48.940915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf1182-d313-4174-af58-31f592b153c6 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:48.947712 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff1efd-3948-477d-b441-9b7f85754d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:48.948378 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff1efd-3948-477d-b441-9b7f85754d39 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:48.948664 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff1efd-3948-477d-b441-9b7f85754d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:48.948939 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff1efd-3948-477d-b441-9b7f85754d39 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:48.949622 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff1efd-3948-477d-b441-9b7f85754d39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:48.951129 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff1efd-3948-477d-b441-9b7f85754d39 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:48.966404 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 500/987] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Jun 24 03:59:48 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 03:59:49.067890 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf1182-d313-4174-af58-31f592b153c6 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:49.093399 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:49.105611 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c519e-610f-4116-a3de-92717d1c452a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:49.106278 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c519e-610f-4116-a3de-92717d1c452a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:49.106592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c519e-610f-4116-a3de-92717d1c452a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:49.108955 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c519e-610f-4116-a3de-92717d1c452a None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:49.108955 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c519e-610f-4116-a3de-92717d1c452a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:49.108955 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c519e-610f-4116-a3de-92717d1c452a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:49.109492 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c519e-610f-4116-a3de-92717d1c452a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:49.125416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 488/988] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:49.132037 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff1efd-3948-477d-b441-9b7f85754d39 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:49.146166 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:49.236621 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff1efd-3948-477d-b441-9b7f85754d39 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:49.270711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de4fcc-ff06-4695-8060-2c314b55bad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:49.271372 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de4fcc-ff06-4695-8060-2c314b55bad4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:49.271660 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de4fcc-ff06-4695-8060-2c314b55bad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:49.271941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de4fcc-ff06-4695-8060-2c314b55bad4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:49.272773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de4fcc-ff06-4695-8060-2c314b55bad4 None admin] RBAC: Adding query filter params (role.id=254f32a34d7b4de7b5c039848b8153d6, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=98f58f91d17949198e66883d234a9e40) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:49.273090 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de4fcc-ff06-4695-8060-2c314b55bad4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:49.274549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de4fcc-ff06-4695-8060-2c314b55bad4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:49.286413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 501/989] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:59:49 2022] GET /identity/v3/role_assignments?user.id=98f58f91d17949198e66883d234a9e40&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=254f32a34d7b4de7b5c039848b8153d6 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:49.606292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a666e7b-0e1b-4873-9bec-f6fc6d0a91c6 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:49.606728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a666e7b-0e1b-4873-9bec-f6fc6d0a91c6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:49.606907 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a666e7b-0e1b-4873-9bec-f6fc6d0a91c6 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:49.608424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 489/990] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:49.613793 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4c34a-9c0c-4bb2-a44f-514a51b728b2 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:49.614090 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4c34a-9c0c-4bb2-a44f-514a51b728b2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:49.614396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4c34a-9c0c-4bb2-a44f-514a51b728b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:49.632009 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5ec4c34a-9c0c-4bb2-a44f-514a51b728b2 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:49.633210 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5ec4c34a-9c0c-4bb2-a44f-514a51b728b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:49.700962 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 502/991] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:49 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:49.724337 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a666e7b-0e1b-4873-9bec-f6fc6d0a91c6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:49.827372 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a666e7b-0e1b-4873-9bec-f6fc6d0a91c6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:49.862514 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffff1c-2f65-438f-9b67-98e720b95f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:49.863401 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffff1c-2f65-438f-9b67-98e720b95f8c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:49.863791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffff1c-2f65-438f-9b67-98e720b95f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:49.864170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffff1c-2f65-438f-9b67-98e720b95f8c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:49.864928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ffff1c-2f65-438f-9b67-98e720b95f8c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:49.866410 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ffff1c-2f65-438f-9b67-98e720b95f8c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:49.878928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-f9ffff1c-2f65-438f-9b67-98e720b95f8c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 03:59:49.878928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:49.878928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:49.878928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:49.878928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:49.878928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:49.878928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:49.878928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:49.878928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 03:59:49.878928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:49.878928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:49.878928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:49.878928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:49.878928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:49.878928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:49.878928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:49.878928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:49.880120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:49.880120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:49.880120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:49.880120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:49.880120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:49.880120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:49.880120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:49.880120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:49.880120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:49.880120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:49.880120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:49.880120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:49.880120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:49.880120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:49.880120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:49.880120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:49.881186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:49.881186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:49.881186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:49.881186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:49.881186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:49.881186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:49.881186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:49.881186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:49.881186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:49.881186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:49.881186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:49.881186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:49.881186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 03:59:49.881186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:49.883132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 490/992] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jun 24 03:59:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:49.888859 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4c34a-9c0c-4bb2-a44f-514a51b728b2 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:49.892123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:49.989817 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4c34a-9c0c-4bb2-a44f-514a51b728b2 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:50.025073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05430eac-2a4d-44ff-b947-758e4ccbc873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:50.025892 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05430eac-2a4d-44ff-b947-758e4ccbc873 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:50.026283 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05430eac-2a4d-44ff-b947-758e4ccbc873 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:50.026700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05430eac-2a4d-44ff-b947-758e4ccbc873 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:50.027515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05430eac-2a4d-44ff-b947-758e4ccbc873 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:50.027931 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05430eac-2a4d-44ff-b947-758e4ccbc873 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:50.032538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05430eac-2a4d-44ff-b947-758e4ccbc873 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:50.038890 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 503/993] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jun 24 03:59:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:50.044528 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffff1c-2f65-438f-9b67-98e720b95f8c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:50.100664 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:50.152308 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffff1c-2f65-438f-9b67-98e720b95f8c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:50.185911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5655a-fc4f-43aa-bfbd-5369ebeeffe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:50.186612 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5655a-fc4f-43aa-bfbd-5369ebeeffe5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:50.186908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5655a-fc4f-43aa-bfbd-5369ebeeffe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:50.187189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5655a-fc4f-43aa-bfbd-5369ebeeffe5 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:50.187898 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5655a-fc4f-43aa-bfbd-5369ebeeffe5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:50.188249 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5655a-fc4f-43aa-bfbd-5369ebeeffe5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:50.189629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5655a-fc4f-43aa-bfbd-5369ebeeffe5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:50.194748 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 491/994] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jun 24 03:59:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:50.200401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05430eac-2a4d-44ff-b947-758e4ccbc873 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:50.269020 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:50.303228 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05430eac-2a4d-44ff-b947-758e4ccbc873 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:50.337226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af6bb9-88e0-4fbb-b2e7-3c2027c4d307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:50.337890 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af6bb9-88e0-4fbb-b2e7-3c2027c4d307 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:50.338177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af6bb9-88e0-4fbb-b2e7-3c2027c4d307 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:50.338480 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af6bb9-88e0-4fbb-b2e7-3c2027c4d307 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:50.339290 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af6bb9-88e0-4fbb-b2e7-3c2027c4d307 None admin] RBAC: Adding query filter params (interface=admin, service_id=e7f2e107823d411391004b433f3e068f, region_id=RegionOne) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:50.339611 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af6bb9-88e0-4fbb-b2e7-3c2027c4d307 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:50.340982 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af6bb9-88e0-4fbb-b2e7-3c2027c4d307 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:50.346904 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 504/995] 10.222.0.16 () {56 vars in 1249 bytes} [Fri Jun 24 03:59:50 2022] GET /identity/v3/endpoints?service_id=e7f2e107823d411391004b433f3e068f&interface=admin®ion_id=RegionOne => generated 193 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:50.352258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5655a-fc4f-43aa-bfbd-5369ebeeffe5 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:50.459383 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5655a-fc4f-43aa-bfbd-5369ebeeffe5 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:50.472910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:50.495167 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e377907f-22e7-4c1b-934b-69d9eb5645fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:50.495828 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377907f-22e7-4c1b-934b-69d9eb5645fb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:50.496110 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377907f-22e7-4c1b-934b-69d9eb5645fb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:50.496372 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e377907f-22e7-4c1b-934b-69d9eb5645fb None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:50.497048 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e377907f-22e7-4c1b-934b-69d9eb5645fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:50.498435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e377907f-22e7-4c1b-934b-69d9eb5645fb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:50.510568 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 492/996] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Jun 24 03:59:50 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 03:59:51.032187 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eba500-fcd5-47d9-a872-4ebeb00d8b2c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:51.032187 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eba500-fcd5-47d9-a872-4ebeb00d8b2c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:51.032187 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eba500-fcd5-47d9-a872-4ebeb00d8b2c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:51.032187 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 505/997] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:51.037336 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b0340-ef36-494b-88bd-1a8f1dd4ccac None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:51.037743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b0340-ef36-494b-88bd-1a8f1dd4ccac None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:51.037945 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b0340-ef36-494b-88bd-1a8f1dd4ccac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:51.055299 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3e3b0340-ef36-494b-88bd-1a8f1dd4ccac None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:51.056497 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3e3b0340-ef36-494b-88bd-1a8f1dd4ccac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:51.126627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 493/998] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:51.148845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eba500-fcd5-47d9-a872-4ebeb00d8b2c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:51.256248 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eba500-fcd5-47d9-a872-4ebeb00d8b2c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:51.285893 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:51.291555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3394f5-b01f-436d-98be-940952c135b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:51.292350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3394f5-b01f-436d-98be-940952c135b5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:51.292631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3394f5-b01f-436d-98be-940952c135b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:51.292904 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3394f5-b01f-436d-98be-940952c135b5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:51.293554 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3394f5-b01f-436d-98be-940952c135b5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:51.294983 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3394f5-b01f-436d-98be-940952c135b5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:51.306534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-bc3394f5-b01f-436d-98be-940952c135b5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 03:59:51.306534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:51.306534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:51.306534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:51.306534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:51.306534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:51.306534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:51.306534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:51.306534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 03:59:51.306534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:51.306534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:51.306534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:51.306534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:51.306534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:51.306534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:51.306534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:51.306534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:51.307592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:51.307592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:51.307592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:51.307592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:51.307592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:51.307592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:51.307592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:51.307592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:51.307592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:51.307592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:51.307592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:51.307592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:51.307592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:51.307592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:51.307592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:51.307592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:51.308533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:51.308533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:51.308533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:51.308533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:51.308533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:51.308533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:51.308533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:51.308533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:51.308533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:51.308533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:51.308533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:51.308533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:51.308533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 03:59:51.308533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:51.308533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 506/999] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:51.314174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b0340-ef36-494b-88bd-1a8f1dd4ccac None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:51.383024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c963d50-5c77-4045-a781-45ae6c40b884 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:51.383311 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c963d50-5c77-4045-a781-45ae6c40b884 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:51.383586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c963d50-5c77-4045-a781-45ae6c40b884 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:51.385075 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 507/1000] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 03:59:51.389757 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a415ac-8438-4f61-9d65-be375af297f6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:51.390046 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a415ac-8438-4f61-9d65-be375af297f6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:51.390328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a415ac-8438-4f61-9d65-be375af297f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:51.407583 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-41a415ac-8438-4f61-9d65-be375af297f6 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:51.408782 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-41a415ac-8438-4f61-9d65-be375af297f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:51.420083 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b0340-ef36-494b-88bd-1a8f1dd4ccac None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:51.428456 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:51.457363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c22fa-24ed-4445-a89e-bed71497e516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:51.458020 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c22fa-24ed-4445-a89e-bed71497e516 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:51.458306 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c22fa-24ed-4445-a89e-bed71497e516 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:51.458616 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c22fa-24ed-4445-a89e-bed71497e516 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:51.459386 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c22fa-24ed-4445-a89e-bed71497e516 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:51.459878 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c22fa-24ed-4445-a89e-bed71497e516 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:51.461424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c22fa-24ed-4445-a89e-bed71497e516 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:51.471526 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 494/1001] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jun 24 03:59:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:51.473011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c22fa-24ed-4445-a89e-bed71497e516 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:51.486997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 508/1002] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 03:59:51.510040 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a415ac-8438-4f61-9d65-be375af297f6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:51.580136 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c22fa-24ed-4445-a89e-bed71497e516 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:51.615087 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc064fa2-2920-40f5-b1f5-65924ba0b9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:51.615747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc064fa2-2920-40f5-b1f5-65924ba0b9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:51.616027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc064fa2-2920-40f5-b1f5-65924ba0b9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:51.616302 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc064fa2-2920-40f5-b1f5-65924ba0b9c1 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:51.617009 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc064fa2-2920-40f5-b1f5-65924ba0b9c1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:51.617312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc064fa2-2920-40f5-b1f5-65924ba0b9c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:51.618711 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc064fa2-2920-40f5-b1f5-65924ba0b9c1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:51.619921 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a415ac-8438-4f61-9d65-be375af297f6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:51.624021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 495/1003] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Jun 24 03:59:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:51.629521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc064fa2-2920-40f5-b1f5-65924ba0b9c1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:51.651124 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:51.661246 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a02fad-0a3f-4cff-b9e2-7b367b40849f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:51.662040 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a02fad-0a3f-4cff-b9e2-7b367b40849f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:51.662330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a02fad-0a3f-4cff-b9e2-7b367b40849f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:51.662643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a02fad-0a3f-4cff-b9e2-7b367b40849f None admin] PATH_INFO: `/v3/services/image` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:51.664055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a02fad-0a3f-4cff-b9e2-7b367b40849f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:51.664882 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a02fad-0a3f-4cff-b9e2-7b367b40849f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:51.666277 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:51.677024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-60a02fad-0a3f-4cff-b9e2-7b367b40849f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 03:59:51.677024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:51.677024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:51.677024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:51.677024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:51.677024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:51.677024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:51.677024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:51.677024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 03:59:51.677024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:51.677024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:51.677024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:51.677024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:51.677024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:51.677024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:51.677024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:51.677024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:51.678115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:51.678115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:51.678115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:51.678115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:51.678115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:51.678115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:51.678115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:51.678115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:51.678115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:51.678115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:51.678115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:51.678115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:51.678115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:51.678115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:51.678115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:51.678115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:51.679030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:51.679030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:51.679030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:51.679030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:51.679030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:51.679030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:51.679030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:51.679030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:51.679030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:51.679030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:51.679030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:51.679030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:51.679030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 03:59:51.679030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:51.679030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 509/1004] 10.222.0.16 () {56 vars in 1105 bytes} [Fri Jun 24 03:59:51 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:51.684056 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a02fad-0a3f-4cff-b9e2-7b367b40849f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:51.739953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc064fa2-2920-40f5-b1f5-65924ba0b9c1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:51.776379 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4bec9a-4edc-40f8-a816-208824082331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:51.777151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bec9a-4edc-40f8-a816-208824082331 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:51.777435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bec9a-4edc-40f8-a816-208824082331 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:51.777827 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4bec9a-4edc-40f8-a816-208824082331 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:51.778604 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bec9a-4edc-40f8-a816-208824082331 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:51.780091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4bec9a-4edc-40f8-a816-208824082331 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:51.784081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a02fad-0a3f-4cff-b9e2-7b367b40849f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:51.817098 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400d430-c2cc-4dbc-b6e7-0e79d53856f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:51.817756 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400d430-c2cc-4dbc-b6e7-0e79d53856f1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:51.818044 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400d430-c2cc-4dbc-b6e7-0e79d53856f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:51.818324 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400d430-c2cc-4dbc-b6e7-0e79d53856f1 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:51.819202 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400d430-c2cc-4dbc-b6e7-0e79d53856f1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:51.819441 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400d430-c2cc-4dbc-b6e7-0e79d53856f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:51.820802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400d430-c2cc-4dbc-b6e7-0e79d53856f1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:51.825597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 510/1005] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:51.831811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400d430-c2cc-4dbc-b6e7-0e79d53856f1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:51.884817 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 496/1006] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Jun 24 03:59:51 2022] POST /identity/v3/endpoints => generated 334 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 03:59:51.895180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:51.939060 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400d430-c2cc-4dbc-b6e7-0e79d53856f1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:51.983039 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cfed8-b08e-4251-ae2d-ecb5ea19fcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:51.983713 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cfed8-b08e-4251-ae2d-ecb5ea19fcab None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:51.983995 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cfed8-b08e-4251-ae2d-ecb5ea19fcab None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:51.984268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cfed8-b08e-4251-ae2d-ecb5ea19fcab None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:51.985580 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68cfed8-b08e-4251-ae2d-ecb5ea19fcab None admin] RBAC: Adding query filter params (type=image) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:51.986054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68cfed8-b08e-4251-ae2d-ecb5ea19fcab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:51.987756 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68cfed8-b08e-4251-ae2d-ecb5ea19fcab None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:51.993029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 511/1007] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:51 2022] GET /identity/v3/services?type=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:52.596929 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9606e-1a47-4c71-8658-43cc6012a099 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:52.597825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9606e-1a47-4c71-8658-43cc6012a099 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:52.598286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9606e-1a47-4c71-8658-43cc6012a099 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:52.600123 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 497/1008] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:52.605051 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbd76c-1318-408c-9d42-56e99147deab None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:52.605446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbd76c-1318-408c-9d42-56e99147deab None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:52.605613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbd76c-1318-408c-9d42-56e99147deab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:52.622795 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ddfbd76c-1318-408c-9d42-56e99147deab None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:52.623987 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ddfbd76c-1318-408c-9d42-56e99147deab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:52.694003 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 512/1009] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:52 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:52.717503 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9606e-1a47-4c71-8658-43cc6012a099 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:52.814411 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:52.824993 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9606e-1a47-4c71-8658-43cc6012a099 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:52.860280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e607e-88f2-4eba-b01a-2e91bfcb3821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:52.861006 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e607e-88f2-4eba-b01a-2e91bfcb3821 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:52.861287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e607e-88f2-4eba-b01a-2e91bfcb3821 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:52.861563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e607e-88f2-4eba-b01a-2e91bfcb3821 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:52.862215 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e607e-88f2-4eba-b01a-2e91bfcb3821 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:52.863613 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e607e-88f2-4eba-b01a-2e91bfcb3821 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:52.877074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-b38e607e-88f2-4eba-b01a-2e91bfcb3821 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 03:59:52.877074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:52.877074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:52.877074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:52.877074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:52.877074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:52.877074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:52.877074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:52.877074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 03:59:52.877074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:52.877074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:52.877074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:52.877074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:52.877074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:52.877074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:52.877074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:52.877074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:52.878270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:52.878270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:52.878270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:52.878270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:52.878270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:52.878270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:52.878270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:52.878270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:52.878270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:52.878270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:52.878270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:52.878270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:52.878270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:52.878270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:52.878270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:52.878270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:52.879155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:52.879155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:52.879155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:52.879155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:52.879155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:52.879155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:52.879155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:52.879155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:52.879155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:52.879155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:52.879155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:52.879155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:52.879155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 03:59:52.879155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:52.887222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 498/1010] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Jun 24 03:59:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:52.888029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbd76c-1318-408c-9d42-56e99147deab None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:52.934890 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:52.991842 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbd76c-1318-408c-9d42-56e99147deab None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:53.027677 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ccfe5-c8ee-4ad6-829d-148e0b214a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:53.028387 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ccfe5-c8ee-4ad6-829d-148e0b214a49 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:53.028679 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ccfe5-c8ee-4ad6-829d-148e0b214a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:53.032522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ccfe5-c8ee-4ad6-829d-148e0b214a49 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:53.032522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ccfe5-c8ee-4ad6-829d-148e0b214a49 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:53.032522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ccfe5-c8ee-4ad6-829d-148e0b214a49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:53.032522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ccfe5-c8ee-4ad6-829d-148e0b214a49 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:53.043429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 513/1011] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jun 24 03:59:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:53.044890 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e607e-88f2-4eba-b01a-2e91bfcb3821 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:53.152761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e607e-88f2-4eba-b01a-2e91bfcb3821 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:53.186986 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165b198f-0930-49a0-a4bb-4a1e24001d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:53.187656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b198f-0930-49a0-a4bb-4a1e24001d39 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:53.187967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b198f-0930-49a0-a4bb-4a1e24001d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:53.188247 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165b198f-0930-49a0-a4bb-4a1e24001d39 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:53.188960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b198f-0930-49a0-a4bb-4a1e24001d39 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:53.189268 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b198f-0930-49a0-a4bb-4a1e24001d39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:53.190688 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165b198f-0930-49a0-a4bb-4a1e24001d39 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:53.200166 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 499/1012] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Jun 24 03:59:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:53.201220 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ccfe5-c8ee-4ad6-829d-148e0b214a49 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:53.305992 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ccfe5-c8ee-4ad6-829d-148e0b214a49 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:53.315179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:53.340154 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86085f05-c23d-4810-b361-cd15c888b624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:53.340989 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86085f05-c23d-4810-b361-cd15c888b624 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:53.341582 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86085f05-c23d-4810-b361-cd15c888b624 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:53.341880 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86085f05-c23d-4810-b361-cd15c888b624 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:53.342671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86085f05-c23d-4810-b361-cd15c888b624 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:53.344110 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86085f05-c23d-4810-b361-cd15c888b624 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:53.397928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 514/1013] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Jun 24 03:59:53 2022] POST /identity/v3/endpoints => generated 325 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 03:59:54.068396 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbb3e6-aebe-4c28-a3fc-4d581113e2af None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:54.068896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbb3e6-aebe-4c28-a3fc-4d581113e2af None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:54.068972 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbb3e6-aebe-4c28-a3fc-4d581113e2af None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:54.070529 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 500/1014] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:54.074228 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493712b-02f8-4cc1-be46-78000e86c373 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:54.074676 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493712b-02f8-4cc1-be46-78000e86c373 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:54.074853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493712b-02f8-4cc1-be46-78000e86c373 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:54.075108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5451d-cf43-4a6c-ba94-57bd07957581 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:54.075401 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5451d-cf43-4a6c-ba94-57bd07957581 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:54.075681 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5451d-cf43-4a6c-ba94-57bd07957581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:54.076295 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 515/1015] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:54.081813 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bc6a3-f276-4580-bfe5-4d774fbaad57 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:54.082100 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bc6a3-f276-4580-bfe5-4d774fbaad57 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:54.082397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146bc6a3-f276-4580-bfe5-4d774fbaad57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:54.093347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-67d5451d-cf43-4a6c-ba94-57bd07957581 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:54.095381 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-67d5451d-cf43-4a6c-ba94-57bd07957581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:54.099617 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-146bc6a3-f276-4580-bfe5-4d774fbaad57 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:54.100777 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-146bc6a3-f276-4580-bfe5-4d774fbaad57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:54.110364 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:54.177261 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 501/1016] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:54 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 03:59:54.177856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 516/1017] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:54 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 03:59:54.200679 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5451d-cf43-4a6c-ba94-57bd07957581 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:54.201540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bc6a3-f276-4580-bfe5-4d774fbaad57 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:54.303595 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:54.310123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146bc6a3-f276-4580-bfe5-4d774fbaad57 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:54.312239 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5451d-cf43-4a6c-ba94-57bd07957581 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:54.349197 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e633600-70e0-4588-b4d1-fed4fe456ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:54.349492 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe877f-4d6a-4f3c-b977-5957068187c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:54.350166 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe877f-4d6a-4f3c-b977-5957068187c4 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:54.350480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe877f-4d6a-4f3c-b977-5957068187c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:54.350761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe877f-4d6a-4f3c-b977-5957068187c4 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:54.351087 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e633600-70e0-4588-b4d1-fed4fe456ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:54.351494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe877f-4d6a-4f3c-b977-5957068187c4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:54.351743 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e633600-70e0-4588-b4d1-fed4fe456ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:54.352202 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e633600-70e0-4588-b4d1-fed4fe456ea5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:54.352855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe877f-4d6a-4f3c-b977-5957068187c4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:54.362840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e633600-70e0-4588-b4d1-fed4fe456ea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:54.371286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-3e633600-70e0-4588-b4d1-fed4fe456ea5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 03:59:54.371286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:54.371286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:54.371286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:54.371286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:54.371286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:54.371286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:54.371286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:54.371286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:54.371286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:54.371286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:54.371286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:54.371286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:54.371286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:54.371286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:54.371286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:54.372273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:54.372273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:54.372273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:54.372273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:54.372273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:54.372273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:54.372273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:54.372273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:54.372273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:54.372273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:54.372273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:54.372273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:54.372273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:54.372273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:54.372273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:54.372273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:54.373115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:54.373115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:54.373115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:54.373115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:54.373115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:54.373115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:54.373115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:54.373115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:54.373115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:54.373115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:54.373115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 03:59:54.373115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:54.373115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 517/1018] 10.222.0.16 () {56 vars in 1120 bytes} [Fri Jun 24 03:59:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:54.378894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e633600-70e0-4588-b4d1-fed4fe456ea5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:54.436487 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 502/1019] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jun 24 03:59:54 2022] POST /identity/v3/services => generated 242 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 03:59:54.484666 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e633600-70e0-4588-b4d1-fed4fe456ea5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:54.531577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadb775-9762-4a32-b77c-f04b61786379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:54.532343 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadb775-9762-4a32-b77c-f04b61786379 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:54.532776 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadb775-9762-4a32-b77c-f04b61786379 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:54.533091 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadb775-9762-4a32-b77c-f04b61786379 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:54.533855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadb775-9762-4a32-b77c-f04b61786379 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:54.534297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadb775-9762-4a32-b77c-f04b61786379 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:54.536523 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadb775-9762-4a32-b77c-f04b61786379 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:54.549470 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 518/1020] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Jun 24 03:59:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:54.557965 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe877f-4d6a-4f3c-b977-5957068187c4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:54.565148 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:54.682209 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe877f-4d6a-4f3c-b977-5957068187c4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:54.717066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d42e2-57c7-49bb-adbf-85395349a5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:54.717847 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d42e2-57c7-49bb-adbf-85395349a5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:54.718134 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d42e2-57c7-49bb-adbf-85395349a5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:54.718429 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d42e2-57c7-49bb-adbf-85395349a5d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:54.729460 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d42e2-57c7-49bb-adbf-85395349a5d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:54.730934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d42e2-57c7-49bb-adbf-85395349a5d3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:54.749003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-748d42e2-57c7-49bb-adbf-85395349a5d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:54.749003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:54.749003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:54.749003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:54.749003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:54.749003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:54.749003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:54.749003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:54.749003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:54.749003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:54.749003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:54.749003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:54.749003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:54.749003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:54.749003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:54.749909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:54.749909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:54.749909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:54.749909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:54.749909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:54.749909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:54.749909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:54.749909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:54.749909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:54.749909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:54.749909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:54.749909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:54.749909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:54.749909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:54.749909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:54.749909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:54.749909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:54.751435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:54.751435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:54.751435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:54.751435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:54.751663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 503/1021] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:54.757512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadb775-9762-4a32-b77c-f04b61786379 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:54.773227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:54.863486 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadb775-9762-4a32-b77c-f04b61786379 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:54.900412 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324ce8c-dbc7-4407-b35c-f80e3d78daa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:54.901379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324ce8c-dbc7-4407-b35c-f80e3d78daa6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:54.901799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324ce8c-dbc7-4407-b35c-f80e3d78daa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:54.902223 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324ce8c-dbc7-4407-b35c-f80e3d78daa6 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:54.903440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324ce8c-dbc7-4407-b35c-f80e3d78daa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:54.903882 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324ce8c-dbc7-4407-b35c-f80e3d78daa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:54.905967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324ce8c-dbc7-4407-b35c-f80e3d78daa6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:54.927523 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 519/1022] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:54.929207 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d42e2-57c7-49bb-adbf-85395349a5d3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:54.998002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:55.049194 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d42e2-57c7-49bb-adbf-85395349a5d3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:55.085948 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6cb38-3e8e-488a-9a0b-18cb3204c85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:55.086770 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6cb38-3e8e-488a-9a0b-18cb3204c85d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:55.087059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6cb38-3e8e-488a-9a0b-18cb3204c85d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:55.087343 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6cb38-3e8e-488a-9a0b-18cb3204c85d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:55.095585 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6cb38-3e8e-488a-9a0b-18cb3204c85d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:55.097204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6cb38-3e8e-488a-9a0b-18cb3204c85d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:55.111880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-d9a6cb38-3e8e-488a-9a0b-18cb3204c85d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 03:59:55.111880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:55.111880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:55.111880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:55.111880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:55.111880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:55.111880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:55.111880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:55.111880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:55.111880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:55.111880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:55.111880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:55.111880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:55.111880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:55.111880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:55.111880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:55.112883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:55.112883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:55.112883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:55.112883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:55.112883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:55.112883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:55.112883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:55.112883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:55.112883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:55.112883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:55.112883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:55.112883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:55.112883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:55.112883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:55.112883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:55.112883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:55.112883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:55.112883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:55.113912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:55.113912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:55.113912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:55.113912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:55.113912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:55.113912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:55.113912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:55.113912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:55.113912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 03:59:55.113912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:55.115232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 504/1023] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Jun 24 03:59:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:55.119077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324ce8c-dbc7-4407-b35c-f80e3d78daa6 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:55.193327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:55.224420 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324ce8c-dbc7-4407-b35c-f80e3d78daa6 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:55.268979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50edf95-2252-4ce6-ad80-1e6ecaeefd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:55.269665 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50edf95-2252-4ce6-ad80-1e6ecaeefd6d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:55.269954 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50edf95-2252-4ce6-ad80-1e6ecaeefd6d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:55.270236 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50edf95-2252-4ce6-ad80-1e6ecaeefd6d None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:55.271117 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50edf95-2252-4ce6-ad80-1e6ecaeefd6d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:55.271430 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50edf95-2252-4ce6-ad80-1e6ecaeefd6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:55.272888 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50edf95-2252-4ce6-ad80-1e6ecaeefd6d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:55.297773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 520/1024] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Jun 24 03:59:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:55.299988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6cb38-3e8e-488a-9a0b-18cb3204c85d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:55.402245 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6cb38-3e8e-488a-9a0b-18cb3204c85d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:55.418500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:55.437006 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20f257-17eb-439b-bea0-542a17cd6815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:55.437662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20f257-17eb-439b-bea0-542a17cd6815 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:55.438556 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20f257-17eb-439b-bea0-542a17cd6815 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:55.438947 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20f257-17eb-439b-bea0-542a17cd6815 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:55.474363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d7f3d-c240-470c-98c0-5a411fc4e5c8 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:55.474734 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d7f3d-c240-470c-98c0-5a411fc4e5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:55.474964 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d7f3d-c240-470c-98c0-5a411fc4e5c8 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:55.476494 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 521/1025] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:55.481150 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5ae15-5986-4ca1-b7ab-57e68c403d86 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:55.481437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5ae15-5986-4ca1-b7ab-57e68c403d86 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:55.481726 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5ae15-5986-4ca1-b7ab-57e68c403d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:55.499715 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5ae5ae15-5986-4ca1-b7ab-57e68c403d86 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:55.501661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5ae5ae15-5986-4ca1-b7ab-57e68c403d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:55.513645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20f257-17eb-439b-bea0-542a17cd6815 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:55.515136 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20f257-17eb-439b-bea0-542a17cd6815 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:55.578434 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 522/1026] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 03:59:55.599827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5ae15-5986-4ca1-b7ab-57e68c403d86 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:55.622622 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 505/1027] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 03:59:55 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 03:59:55.629365 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20f257-17eb-439b-bea0-542a17cd6815 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:55.648499 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:55.666006 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:55.710030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5ae15-5986-4ca1-b7ab-57e68c403d86 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:55.737882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20f257-17eb-439b-bea0-542a17cd6815 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:55.748072 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b3a90-b3eb-4bfe-ad4c-8709e63550ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:55.748858 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b3a90-b3eb-4bfe-ad4c-8709e63550ad None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:55.749146 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b3a90-b3eb-4bfe-ad4c-8709e63550ad None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:55.749459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157b3a90-b3eb-4bfe-ad4c-8709e63550ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:55.750641 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b3a90-b3eb-4bfe-ad4c-8709e63550ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:55.752074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157b3a90-b3eb-4bfe-ad4c-8709e63550ad None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:55.753848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 523/1028] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:55.759967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b3a90-b3eb-4bfe-ad4c-8709e63550ad None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:55.773466 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a7e89-fc1f-49ce-9019-acbd4664733d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:55.774255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a7e89-fc1f-49ce-9019-acbd4664733d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:55.774575 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a7e89-fc1f-49ce-9019-acbd4664733d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:55.774856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a7e89-fc1f-49ce-9019-acbd4664733d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:55.785632 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a7e89-fc1f-49ce-9019-acbd4664733d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:55.787109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a7e89-fc1f-49ce-9019-acbd4664733d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:55.806061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-5a5a7e89-fc1f-49ce-9019-acbd4664733d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:55.806061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:55.806061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:55.806061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:55.806061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:55.806061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:55.806061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:55.806061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:55.806061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:55.806061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:55.806061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:55.806061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:55.806061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:55.806061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:55.806061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:55.807174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:55.807174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:55.807174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:55.807174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:55.807174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:55.807174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:55.807174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:55.807174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:55.807174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:55.807174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:55.807174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:55.807174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:55.807174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:55.807174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:55.807174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:55.807174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:55.807174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:55.808115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:55.808115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:55.808115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:55.808115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:55.808115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 506/1029] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:55.812806 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a7e89-fc1f-49ce-9019-acbd4664733d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:55.877333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157b3a90-b3eb-4bfe-ad4c-8709e63550ad None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:55.884279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:55.904440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:55.913704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f67cc9-bb46-42f9-b5ca-5b2a82d7e528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:55.914393 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f67cc9-bb46-42f9-b5ca-5b2a82d7e528 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:55.914739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f67cc9-bb46-42f9-b5ca-5b2a82d7e528 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:55.915021 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f67cc9-bb46-42f9-b5ca-5b2a82d7e528 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:55.915665 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f67cc9-bb46-42f9-b5ca-5b2a82d7e528 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:55.917347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f67cc9-bb46-42f9-b5ca-5b2a82d7e528 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:55.920768 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a7e89-fc1f-49ce-9019-acbd4664733d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:55.955307 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b0c29-b65d-4d91-8c25-60d61431f680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:55.956132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b0c29-b65d-4d91-8c25-60d61431f680 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:55.956355 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b0c29-b65d-4d91-8c25-60d61431f680 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:55.956637 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b0c29-b65d-4d91-8c25-60d61431f680 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:55.957353 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b0c29-b65d-4d91-8c25-60d61431f680 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:55.957659 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b0c29-b65d-4d91-8c25-60d61431f680 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:55.959054 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b0c29-b65d-4d91-8c25-60d61431f680 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:55.971322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 507/1030] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:55.976866 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b0c29-b65d-4d91-8c25-60d61431f680 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:56.014933 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 524/1031] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jun 24 03:59:55 2022] POST /identity/v3/projects => generated 311 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 03:59:56.093565 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b0c29-b65d-4d91-8c25-60d61431f680 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:56.122988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:56.131699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e48c1-56a9-40de-aa68-cec71c1ccd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:56.132529 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e48c1-56a9-40de-aa68-cec71c1ccd3a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:56.132818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e48c1-56a9-40de-aa68-cec71c1ccd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:56.133102 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e48c1-56a9-40de-aa68-cec71c1ccd3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:56.140908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e48c1-56a9-40de-aa68-cec71c1ccd3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:56.142979 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e48c1-56a9-40de-aa68-cec71c1ccd3a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:56.158870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-f53e48c1-56a9-40de-aa68-cec71c1ccd3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:56.158870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:56.158870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:56.158870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:56.158870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:56.158870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:56.158870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:56.158870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:56.158870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:56.158870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:56.158870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:56.158870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:56.158870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 03:59:56.158870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 03:59:56.158870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 03:59:56.159850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 03:59:56.159850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:56.159850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:56.159850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:56.159850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:56.159850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:56.159850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:56.159850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:56.159850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:56.159850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:56.159850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:56.159850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:56.159850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:56.159850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:56.159850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:56.159850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:56.159850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:56.160802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 03:59:56.160802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 03:59:56.160802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 03:59:56.160802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 03:59:56.160802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 03:59:56.160802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 03:59:56.160802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 03:59:56.160802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:56.160802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 508/1032] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 03:59:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:56.166152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f67cc9-bb46-42f9-b5ca-5b2a82d7e528 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:56.277546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f67cc9-bb46-42f9-b5ca-5b2a82d7e528 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:56.314909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c666b4d-eef6-41a4-8178-eb5aa6911624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:56.315854 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c666b4d-eef6-41a4-8178-eb5aa6911624 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:56.316289 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c666b4d-eef6-41a4-8178-eb5aa6911624 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:56.316731 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c666b4d-eef6-41a4-8178-eb5aa6911624 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:56.317673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c666b4d-eef6-41a4-8178-eb5aa6911624 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:56.318133 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c666b4d-eef6-41a4-8178-eb5aa6911624 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:56.319738 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c666b4d-eef6-41a4-8178-eb5aa6911624 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:56.333184 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 525/1033] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 03:59:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:56.338974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e48c1-56a9-40de-aa68-cec71c1ccd3a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:56.444598 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e48c1-56a9-40de-aa68-cec71c1ccd3a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:56.480774 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba87030-cc5f-4afe-bf8b-8f2b278cb97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:56.481430 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba87030-cc5f-4afe-bf8b-8f2b278cb97b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:56.481714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba87030-cc5f-4afe-bf8b-8f2b278cb97b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:56.481992 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba87030-cc5f-4afe-bf8b-8f2b278cb97b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:56.482858 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba87030-cc5f-4afe-bf8b-8f2b278cb97b None admin] RBAC: Adding query filter params (role.id=254f32a34d7b4de7b5c039848b8153d6, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=80b7b3ec60db4de19bbf7d08f111f470) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:56.483178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba87030-cc5f-4afe-bf8b-8f2b278cb97b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:56.484608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba87030-cc5f-4afe-bf8b-8f2b278cb97b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:56.498408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 509/1034] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 03:59:56 2022] GET /identity/v3/role_assignments?user.id=80b7b3ec60db4de19bbf7d08f111f470&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=254f32a34d7b4de7b5c039848b8153d6 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:56.539303 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e366613-4991-4c75-b76e-dc52542e3ac9 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:56.539551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e366613-4991-4c75-b76e-dc52542e3ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:56.539799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e366613-4991-4c75-b76e-dc52542e3ac9 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:56.541429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 526/1035] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:56.550683 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb8af2-e9e6-4ac9-868b-230ae09db89d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:56.551304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb8af2-e9e6-4ac9-868b-230ae09db89d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:56.551758 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fb8af2-e9e6-4ac9-868b-230ae09db89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:56.569705 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-50fb8af2-e9e6-4ac9-868b-230ae09db89d None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:56.570944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-50fb8af2-e9e6-4ac9-868b-230ae09db89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:56.623220 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:56.648508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 510/1036] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:56.677619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e366613-4991-4c75-b76e-dc52542e3ac9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:56.723663 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:56.784703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e366613-4991-4c75-b76e-dc52542e3ac9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:56.819969 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143be1c1-d574-4a20-9f4e-f80f374dfc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:56.820684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143be1c1-d574-4a20-9f4e-f80f374dfc7f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:56.820977 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143be1c1-d574-4a20-9f4e-f80f374dfc7f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:56.821254 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143be1c1-d574-4a20-9f4e-f80f374dfc7f None admin] PATH_INFO: `/v3/services/image` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:56.821912 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143be1c1-d574-4a20-9f4e-f80f374dfc7f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:56.823326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143be1c1-d574-4a20-9f4e-f80f374dfc7f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:56.835791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-143be1c1-d574-4a20-9f4e-f80f374dfc7f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 03:59:56.835791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:56.835791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:56.835791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:56.835791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:56.835791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:56.835791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:56.835791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:56.835791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 03:59:56.835791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:56.835791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:56.835791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:56.835791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:56.835791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:56.835791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:56.835791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:56.835791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:56.836829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:56.836829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:56.836829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:56.836829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:56.836829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:56.836829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:56.836829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:56.836829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:56.836829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:56.836829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:56.836829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:56.836829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:56.836829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:56.836829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:56.836829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:56.836829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:56.842695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:56.842695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:56.842695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:56.842695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:56.842695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:56.842695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:56.842695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:56.842695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:56.842695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:56.842695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:56.842695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:56.842695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:56.842695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 03:59:56.842695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:56.842695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 527/1037] 10.222.0.16 () {56 vars in 1105 bytes} [Fri Jun 24 03:59:56 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:56.845045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb8af2-e9e6-4ac9-868b-230ae09db89d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:56.945632 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fb8af2-e9e6-4ac9-868b-230ae09db89d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:56.980561 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36d080-0d60-4f2b-be37-d9f01fc1adc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:56.981228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36d080-0d60-4f2b-be37-d9f01fc1adc3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:56.981513 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36d080-0d60-4f2b-be37-d9f01fc1adc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:56.981791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36d080-0d60-4f2b-be37-d9f01fc1adc3 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:56.982528 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36d080-0d60-4f2b-be37-d9f01fc1adc3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:56.982842 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36d080-0d60-4f2b-be37-d9f01fc1adc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:56.984227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36d080-0d60-4f2b-be37-d9f01fc1adc3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:56.994780 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 511/1038] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:56.995943 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143be1c1-d574-4a20-9f4e-f80f374dfc7f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:57.108259 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143be1c1-d574-4a20-9f4e-f80f374dfc7f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:57.111072 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:57.143702 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bea1ab-6501-4039-943a-522c5cfdc0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:57.144379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bea1ab-6501-4039-943a-522c5cfdc0ad None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:57.144666 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bea1ab-6501-4039-943a-522c5cfdc0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:57.144942 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bea1ab-6501-4039-943a-522c5cfdc0ad None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:57.145659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bea1ab-6501-4039-943a-522c5cfdc0ad None admin] RBAC: Adding query filter params (type=image) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:57.145970 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bea1ab-6501-4039-943a-522c5cfdc0ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:57.147375 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bea1ab-6501-4039-943a-522c5cfdc0ad None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:57.158361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 528/1039] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:57.159637 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36d080-0d60-4f2b-be37-d9f01fc1adc3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:57.208782 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:57.266841 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36d080-0d60-4f2b-be37-d9f01fc1adc3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:57.301233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14622f-c602-4789-8bfd-6436ff28d312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:57.301893 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14622f-c602-4789-8bfd-6436ff28d312 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:57.302178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14622f-c602-4789-8bfd-6436ff28d312 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:57.302494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14622f-c602-4789-8bfd-6436ff28d312 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:57.303282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14622f-c602-4789-8bfd-6436ff28d312 None admin] RBAC: Adding query filter params (interface=public, service_id=8b2d1026a9e0450c8a96dc579e18f146, region_id=RegionOne) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:57.303592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14622f-c602-4789-8bfd-6436ff28d312 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:57.304959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14622f-c602-4789-8bfd-6436ff28d312 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:57.315177 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 512/1040] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Jun 24 03:59:57 2022] GET /identity/v3/endpoints?service_id=8b2d1026a9e0450c8a96dc579e18f146&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:57.316432 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bea1ab-6501-4039-943a-522c5cfdc0ad None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:57.417064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bea1ab-6501-4039-943a-522c5cfdc0ad None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:57.450297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f541a4-a3dd-451d-8cd8-20856ea0d05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:57.450986 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f541a4-a3dd-451d-8cd8-20856ea0d05e None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:57.451273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f541a4-a3dd-451d-8cd8-20856ea0d05e None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:57.451554 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f541a4-a3dd-451d-8cd8-20856ea0d05e None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:57.452234 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f541a4-a3dd-451d-8cd8-20856ea0d05e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:57.454142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f541a4-a3dd-451d-8cd8-20856ea0d05e None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:57.466451 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 529/1041] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Jun 24 03:59:57 2022] GET /identity/v3/services => generated 1957 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 03:59:58.154212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a8268-a668-42f3-b78b-8db7433a006b None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:58.154670 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a8268-a668-42f3-b78b-8db7433a006b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:58.154862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52a8268-a668-42f3-b78b-8db7433a006b None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:58.156213 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 513/1042] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:58.162018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b4f4c-bbec-4d4c-9c6d-35623beee1ad None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:58.162319 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b4f4c-bbec-4d4c-9c6d-35623beee1ad None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:58.162637 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b4f4c-bbec-4d4c-9c6d-35623beee1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:58.180588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-789b4f4c-bbec-4d4c-9c6d-35623beee1ad None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:58.181782 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-789b4f4c-bbec-4d4c-9c6d-35623beee1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:58.208753 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:58.254636 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 530/1043] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 03:59:58.275418 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a8268-a668-42f3-b78b-8db7433a006b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:58.380548 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52a8268-a668-42f3-b78b-8db7433a006b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:58.391759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:58.415428 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f010eb-a6fa-4668-bc39-89d2bd72c0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:58.416212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f010eb-a6fa-4668-bc39-89d2bd72c0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:58.416496 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f010eb-a6fa-4668-bc39-89d2bd72c0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:58.416769 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f010eb-a6fa-4668-bc39-89d2bd72c0f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:58.417947 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f010eb-a6fa-4668-bc39-89d2bd72c0f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:58.419470 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f010eb-a6fa-4668-bc39-89d2bd72c0f7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:58.426339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 514/1044] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:58.427610 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b4f4c-bbec-4d4c-9c6d-35623beee1ad None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:58.530394 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b4f4c-bbec-4d4c-9c6d-35623beee1ad None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:58.562520 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7520d-5e4c-408b-9fff-c99d3a04890a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:58.563269 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7520d-5e4c-408b-9fff-c99d3a04890a None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:58.563545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7520d-5e4c-408b-9fff-c99d3a04890a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:58.563914 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c7520d-5e4c-408b-9fff-c99d3a04890a None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:58.564655 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7520d-5e4c-408b-9fff-c99d3a04890a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:58.566122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c7520d-5e4c-408b-9fff-c99d3a04890a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:58.577145 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63526f3-d475-4a46-a113-8c3eca2480d5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:58.577470 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63526f3-d475-4a46-a113-8c3eca2480d5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:58.577702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63526f3-d475-4a46-a113-8c3eca2480d5 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:58.580005 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 515/1045] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:58.585436 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c772358-6661-4000-bca5-05892587cdc0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:58.588857 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c772358-6661-4000-bca5-05892587cdc0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:58.588857 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c772358-6661-4000-bca5-05892587cdc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:58.603415 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1c772358-6661-4000-bca5-05892587cdc0 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:58.604607 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1c772358-6661-4000-bca5-05892587cdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:58.667752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 531/1046] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jun 24 03:59:58 2022] POST /identity/v3/users => generated 285 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 03:59:58.680722 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 516/1047] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 03:59:58.704704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7520d-5e4c-408b-9fff-c99d3a04890a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:58.727075 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:58.814891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c7520d-5e4c-408b-9fff-c99d3a04890a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:58.853239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb9182-27bb-4144-b197-8b5d7b89a88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:58.854844 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb9182-27bb-4144-b197-8b5d7b89a88b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:58.854844 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb9182-27bb-4144-b197-8b5d7b89a88b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:58.854844 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb9182-27bb-4144-b197-8b5d7b89a88b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:58.864733 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb9182-27bb-4144-b197-8b5d7b89a88b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 03:59:58.873297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-08fb9182-27bb-4144-b197-8b5d7b89a88b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 03:59:58.873297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:58.873297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:58.873297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:58.873297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:58.873297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:58.873297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:58.873297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:58.873297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:58.873297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:58.873297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:58.873297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:58.873297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 03:59:58.873297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 03:59:58.873297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 03:59:58.873297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 03:59:58.874327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 03:59:58.874327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 03:59:58.874327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 03:59:58.874327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 03:59:58.874327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:58.874327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:58.874327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:58.874327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:58.874327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:58.874327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:58.874327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:58.874327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:58.874327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:58.874327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:58.874327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:58.874327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:58.875848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:58.875848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:58.875848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:58.875848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:58.875848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 03:59:58.875848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 03:59:58.875848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 03:59:58.875848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 03:59:58.875848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 03:59:58.875848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 03:59:58.875848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 03:59:58.875848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:58.875848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 532/1048] 10.222.0.16 () {56 vars in 1120 bytes} [Fri Jun 24 03:59:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:58.881258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c772358-6661-4000-bca5-05892587cdc0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:58.983307 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c772358-6661-4000-bca5-05892587cdc0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:58.993446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:59.019674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379e9f0-5960-4b10-b05c-934a15c285b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:59.020344 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379e9f0-5960-4b10-b05c-934a15c285b8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:59.020628 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379e9f0-5960-4b10-b05c-934a15c285b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:59.020905 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379e9f0-5960-4b10-b05c-934a15c285b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:59.021655 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379e9f0-5960-4b10-b05c-934a15c285b8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:59.021962 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379e9f0-5960-4b10-b05c-934a15c285b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:59.023384 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379e9f0-5960-4b10-b05c-934a15c285b8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:59.035063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 517/1049] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Jun 24 03:59:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:59.042528 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb9182-27bb-4144-b197-8b5d7b89a88b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:59.123924 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:59.145759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb9182-27bb-4144-b197-8b5d7b89a88b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:59.179928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3211908-e4e5-4dc1-9805-7f447233c0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:59.180718 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3211908-e4e5-4dc1-9805-7f447233c0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:59.180996 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3211908-e4e5-4dc1-9805-7f447233c0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:59.181269 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3211908-e4e5-4dc1-9805-7f447233c0a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:59.192356 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3211908-e4e5-4dc1-9805-7f447233c0a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:59.193809 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3211908-e4e5-4dc1-9805-7f447233c0a5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:59.210855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-c3211908-e4e5-4dc1-9805-7f447233c0a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:59.210855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:59.210855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:59.210855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:59.210855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:59.210855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:59.210855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:59.210855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:59.210855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:59.210855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:59.210855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:59.210855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:59.210855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:59.210855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:59.210855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:59.211839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:59.211839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:59.211839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:59.211839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:59.211839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:59.211839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:59.211839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:59.211839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:59.211839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:59.211839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:59.211839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:59.211839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:59.211839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:59.211839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:59.211839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:59.211839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:59.211839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:59.212750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:59.212750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:59.212750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:59.212750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:59.214452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 533/1050] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 03:59:59.217776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379e9f0-5960-4b10-b05c-934a15c285b8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:59.321606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379e9f0-5960-4b10-b05c-934a15c285b8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:59.358873 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80528575-4bb5-43ca-95aa-00466e8cb84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:59.359537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80528575-4bb5-43ca-95aa-00466e8cb84a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:59.360356 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80528575-4bb5-43ca-95aa-00466e8cb84a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:59.360631 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80528575-4bb5-43ca-95aa-00466e8cb84a None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:59.361451 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80528575-4bb5-43ca-95aa-00466e8cb84a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:59.361762 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80528575-4bb5-43ca-95aa-00466e8cb84a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:59.363214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80528575-4bb5-43ca-95aa-00466e8cb84a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:59.380648 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 518/1051] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 03:59:59.382133 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3211908-e4e5-4dc1-9805-7f447233c0a5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:59.484522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3211908-e4e5-4dc1-9805-7f447233c0a5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:59.513586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:59.519872 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc059c66-896b-47a4-92cd-79f21933479a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:59.520673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc059c66-896b-47a4-92cd-79f21933479a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:59.520963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc059c66-896b-47a4-92cd-79f21933479a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:59.521244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc059c66-896b-47a4-92cd-79f21933479a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:59.529492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc059c66-896b-47a4-92cd-79f21933479a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:59.534723 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc059c66-896b-47a4-92cd-79f21933479a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:59.538819 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3803b38-a1b3-45b2-986e-2fa01df31015 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:59.539164 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3803b38-a1b3-45b2-986e-2fa01df31015 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:59.539590 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3803b38-a1b3-45b2-986e-2fa01df31015 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:59.541132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 519/1052] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 03:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 03:59:59.545597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-dc059c66-896b-47a4-92cd-79f21933479a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 03:59:59.545597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:59.545597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:59.545597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:59.545597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:59.545597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:59.545597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:59.545597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:59.545597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:59.545597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:59.545597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:59.545597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:59.545597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 03:59:59.545597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 03:59:59.545597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 03:59:59.545597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 03:59:59.546855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:59.546855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:59.546855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 03:59:59.546855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:59.546855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 03:59:59.546855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 03:59:59.546855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:59.546855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:59.546855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:59.546855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:59.546855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:59.546855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:59.546855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:59.546855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:59.546855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:59.546855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:59.546855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:59.546855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:59.548762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:59.548762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:59.548762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 03:59:59.548762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 03:59:59.548762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 03:59:59.548762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 03:59:59.548762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 03:59:59.548762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 03:59:59.548762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 03:59:59.548762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:59.549981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84eaff-52b8-4a22-a42a-b1c08e599ae4 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:59.549981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84eaff-52b8-4a22-a42a-b1c08e599ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:59.549981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84eaff-52b8-4a22-a42a-b1c08e599ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:59.552283 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 534/1053] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Jun 24 03:59:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:59.553361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc059c66-896b-47a4-92cd-79f21933479a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:59.565054 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8a84eaff-52b8-4a22-a42a-b1c08e599ae4 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 03:59:59.566242 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8a84eaff-52b8-4a22-a42a-b1c08e599ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 03:59:59.592062 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:59.641015 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 520/1054] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 03:59:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 03:59:59.660512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc059c66-896b-47a4-92cd-79f21933479a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:59.664475 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84eaff-52b8-4a22-a42a-b1c08e599ae4 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:59.699418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f1800-0e74-4b37-b89a-30a3f8790707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:59.700394 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f1800-0e74-4b37-b89a-30a3f8790707 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:59.700844 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f1800-0e74-4b37-b89a-30a3f8790707 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:59.701281 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f1800-0e74-4b37-b89a-30a3f8790707 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:59.702266 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f1800-0e74-4b37-b89a-30a3f8790707 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:59.702746 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f1800-0e74-4b37-b89a-30a3f8790707 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:59.704543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f1800-0e74-4b37-b89a-30a3f8790707 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:59.726914 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 535/1055] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Jun 24 03:59:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:59.732439 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f1800-0e74-4b37-b89a-30a3f8790707 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:59.772770 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84eaff-52b8-4a22-a42a-b1c08e599ae4 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:59.783445 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:59.809067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab70a7-f95e-4e60-95ec-66729c31d976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:59.809861 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab70a7-f95e-4e60-95ec-66729c31d976 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:59.810149 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab70a7-f95e-4e60-95ec-66729c31d976 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:59.810547 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab70a7-f95e-4e60-95ec-66729c31d976 None admin] PATH_INFO: `/v3/services/image` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:59.811216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab70a7-f95e-4e60-95ec-66729c31d976 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:59.812590 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcab70a7-f95e-4e60-95ec-66729c31d976 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:59.826900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-bcab70a7-f95e-4e60-95ec-66729c31d976 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 03:59:59.826900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:59.826900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 03:59:59.826900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 03:59:59.826900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 03:59:59.826900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 03:59:59.826900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 03:59:59.826900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:59.826900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 03:59:59.826900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:59.826900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 03:59:59.826900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:59.826900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:59.826900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:59.826900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:59.826900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:59.826900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:59.828022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:59.828022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:59.828022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:59.828022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:59.828022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:59.828022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:59.828022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 03:59:59.828022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 03:59:59.828022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 03:59:59.828022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 03:59:59.828022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:59.828022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:59.828022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:59.828022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:59.828022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:59.828022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:59.828960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:59.828960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:59.828960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:59.828960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:59.828960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:59.828960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:59.828960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:59.828960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:59.828960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:59.828960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:59.828960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 03:59:59.828960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 03:59:59.828960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 03:59:59.828960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 03:59:59.828960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 521/1056] 10.222.0.16 () {56 vars in 1105 bytes} [Fri Jun 24 03:59:59 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:59.834102 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab70a7-f95e-4e60-95ec-66729c31d976 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:59.840517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f1800-0e74-4b37-b89a-30a3f8790707 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:59.861523 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 03:59:59.880718 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412aeccb-5df8-40ea-8f3a-42692d3bcceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:59.881568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412aeccb-5df8-40ea-8f3a-42692d3bcceb None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:59.881935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412aeccb-5df8-40ea-8f3a-42692d3bcceb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:59.882187 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412aeccb-5df8-40ea-8f3a-42692d3bcceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:59.894068 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412aeccb-5df8-40ea-8f3a-42692d3bcceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:59.895990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412aeccb-5df8-40ea-8f3a-42692d3bcceb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:59.919411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-412aeccb-5df8-40ea-8f3a-42692d3bcceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:59.919411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 03:59:59.919411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 03:59:59.919411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 03:59:59.919411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 03:59:59.919411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 03:59:59.919411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 03:59:59.919411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 03:59:59.919411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 03:59:59.919411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 03:59:59.919411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 03:59:59.919411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 03:59:59.919411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 03:59:59.919411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 03:59:59.919411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 03:59:59.920362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 03:59:59.920362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 03:59:59.920362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 03:59:59.920362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 03:59:59.920362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 03:59:59.920362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 03:59:59.920362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 03:59:59.920362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 03:59:59.920362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 03:59:59.920362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 03:59:59.920362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 03:59:59.920362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 03:59:59.920362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 03:59:59.920362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 03:59:59.920362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 03:59:59.920362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 03:59:59.920362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 03:59:59.926985 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 03:59:59.926985 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 03:59:59.926985 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 03:59:59.926985 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 03:59:59.926985 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 536/1057] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 03:59:59.927628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412aeccb-5df8-40ea-8f3a-42692d3bcceb None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 03:59:59.944166 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab70a7-f95e-4e60-95ec-66729c31d976 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 03:59:59.978545 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176cdb7-934e-4472-8e4e-72ef869c272a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 03:59:59.979199 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176cdb7-934e-4472-8e4e-72ef869c272a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 03:59:59.979505 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176cdb7-934e-4472-8e4e-72ef869c272a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 03:59:59.979763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176cdb7-934e-4472-8e4e-72ef869c272a None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 03:59:59.980468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176cdb7-934e-4472-8e4e-72ef869c272a None admin] RBAC: Adding query filter params (name=image) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 03:59:59.980772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176cdb7-934e-4472-8e4e-72ef869c272a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 03:59:59.982133 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176cdb7-934e-4472-8e4e-72ef869c272a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 03:59:59.988064 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 522/1058] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 03:59:59.993804 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176cdb7-934e-4472-8e4e-72ef869c272a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:00.044892 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412aeccb-5df8-40ea-8f3a-42692d3bcceb None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:00.047044 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:00.082128 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158cace-fd80-4011-ba55-67e0a62ca5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:00.083152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158cace-fd80-4011-ba55-67e0a62ca5ef None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:00.083669 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158cace-fd80-4011-ba55-67e0a62ca5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:00.084078 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158cace-fd80-4011-ba55-67e0a62ca5ef None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:00.085188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158cace-fd80-4011-ba55-67e0a62ca5ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:00.085510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158cace-fd80-4011-ba55-67e0a62ca5ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:00.086924 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158cace-fd80-4011-ba55-67e0a62ca5ef None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:00.091263 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:00.102631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 537/1059] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 03:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:00.103642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158cace-fd80-4011-ba55-67e0a62ca5ef None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:00.106034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176cdb7-934e-4472-8e4e-72ef869c272a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:00.143571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4804f-100a-42b7-99ba-adc88ad4715a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:00.144235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4804f-100a-42b7-99ba-adc88ad4715a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:00.144520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4804f-100a-42b7-99ba-adc88ad4715a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:00.144799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4804f-100a-42b7-99ba-adc88ad4715a None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:00.145616 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4804f-100a-42b7-99ba-adc88ad4715a None admin] RBAC: Adding query filter params (type=image) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:00.145931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4804f-100a-42b7-99ba-adc88ad4715a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:00.147474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4804f-100a-42b7-99ba-adc88ad4715a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:00.158536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 523/1060] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 03:59:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:00.159550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4804f-100a-42b7-99ba-adc88ad4715a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:00.209318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158cace-fd80-4011-ba55-67e0a62ca5ef None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:00.248323 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f103ea0-1c45-4db8-8fcc-4f37e97a0a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:00.249120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f103ea0-1c45-4db8-8fcc-4f37e97a0a21 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:00.249408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f103ea0-1c45-4db8-8fcc-4f37e97a0a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:00.249702 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f103ea0-1c45-4db8-8fcc-4f37e97a0a21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:00.256796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f103ea0-1c45-4db8-8fcc-4f37e97a0a21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:00.258243 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f103ea0-1c45-4db8-8fcc-4f37e97a0a21 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:00.269331 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4804f-100a-42b7-99ba-adc88ad4715a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:00.272608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-6f103ea0-1c45-4db8-8fcc-4f37e97a0a21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 04:00:00.272608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:00:00.272608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:00:00.272608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:00:00.272608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:00:00.272608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:00:00.272608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:00:00.272608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:00:00.272608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:00:00.272608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:00:00.272608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:00:00.272608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:00:00.272608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 04:00:00.272608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 04:00:00.272608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 04:00:00.273536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 04:00:00.273536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:00:00.273536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:00:00.273536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:00:00.273536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:00:00.273536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:00:00.273536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:00:00.273536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:00:00.273536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:00:00.273536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:00:00.273536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:00:00.273536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:00:00.273536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:00:00.273536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:00:00.273536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:00:00.273536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:00:00.273536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:00:00.275071 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 04:00:00.275071 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 04:00:00.275071 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 04:00:00.275071 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 04:00:00.275071 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 04:00:00.275071 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 04:00:00.275071 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 04:00:00.275071 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:00:00.275071 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 538/1061] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 04:00:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 04:00:00.279419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f103ea0-1c45-4db8-8fcc-4f37e97a0a21 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:00.302703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:00.307405 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159cf2d5-850c-468c-9138-c2f399498bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:00.314821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cf2d5-850c-468c-9138-c2f399498bf7 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:00.314821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cf2d5-850c-468c-9138-c2f399498bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:00.314821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159cf2d5-850c-468c-9138-c2f399498bf7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:00.314821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159cf2d5-850c-468c-9138-c2f399498bf7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:00.314821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159cf2d5-850c-468c-9138-c2f399498bf7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:00.384547 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f103ea0-1c45-4db8-8fcc-4f37e97a0a21 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:00.385097 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 524/1062] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Jun 24 04:00:00 2022] POST /identity/v3/endpoints => generated 327 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:00:00.433631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312bf96d-a592-4449-ac46-7c0a914034de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:00.434226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bf96d-a592-4449-ac46-7c0a914034de None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:00.434543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bf96d-a592-4449-ac46-7c0a914034de None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:00.434818 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312bf96d-a592-4449-ac46-7c0a914034de None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:00.435631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312bf96d-a592-4449-ac46-7c0a914034de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:00.435941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312bf96d-a592-4449-ac46-7c0a914034de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:00.437417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312bf96d-a592-4449-ac46-7c0a914034de None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:00.456996 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 539/1063] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 04:00:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:00:00.458758 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159cf2d5-850c-468c-9138-c2f399498bf7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:00.471469 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:00.572061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159cf2d5-850c-468c-9138-c2f399498bf7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:00.611296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2ed5ff-21d5-4597-a276-7b74b181f563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:00.612196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ed5ff-21d5-4597-a276-7b74b181f563 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:00.612526 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ed5ff-21d5-4597-a276-7b74b181f563 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:00.612805 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2ed5ff-21d5-4597-a276-7b74b181f563 None admin] PATH_INFO: `/v3/projects/68d376986e854a4bb334676572340b4c/users/80b7b3ec60db4de19bbf7d08f111f470/roles/254f32a34d7b4de7b5c039848b8153d6` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:00.615006 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ed5ff-21d5-4597-a276-7b74b181f563 None admin] RBAC: Authorizing `identity:create_grant(project_id=68d376986e854a4bb334676572340b4c, user_id=80b7b3ec60db4de19bbf7d08f111f470, role_id=254f32a34d7b4de7b5c039848b8153d6)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:00.616448 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2ed5ff-21d5-4597-a276-7b74b181f563 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:00.726495 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 525/1064] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 04:00:00 2022] PUT /identity/v3/projects/68d376986e854a4bb334676572340b4c/users/80b7b3ec60db4de19bbf7d08f111f470/roles/254f32a34d7b4de7b5c039848b8153d6 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:00:00.811617 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fb0cc-745e-4aa9-9995-7c4832e80504 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:00.812179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fb0cc-745e-4aa9-9995-7c4832e80504 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:00.812179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fb0cc-745e-4aa9-9995-7c4832e80504 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:00.814333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 540/1065] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:00:00.819833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d2fe7-a031-43d6-b029-7349e97db761 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:00.820123 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d2fe7-a031-43d6-b029-7349e97db761 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:00.820411 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d2fe7-a031-43d6-b029-7349e97db761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:00.838392 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-de8d2fe7-a031-43d6-b029-7349e97db761 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:00.839626 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-de8d2fe7-a031-43d6-b029-7349e97db761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:00.911211 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:00.916966 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 526/1066] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:00:00.940921 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0fb0cc-745e-4aa9-9995-7c4832e80504 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:01.025213 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:01.066060 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0fb0cc-745e-4aa9-9995-7c4832e80504 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:01.101794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7989d35-c883-4a07-9b1d-02dfa51101f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:01.102616 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7989d35-c883-4a07-9b1d-02dfa51101f9 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:01.102902 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7989d35-c883-4a07-9b1d-02dfa51101f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:01.103180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7989d35-c883-4a07-9b1d-02dfa51101f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:01.114436 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7989d35-c883-4a07-9b1d-02dfa51101f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 04:00:01.122360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-d7989d35-c883-4a07-9b1d-02dfa51101f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:00:01.122360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:00:01.122360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:00:01.122360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:00:01.122360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:00:01.122360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:00:01.122360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:00:01.122360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:00:01.122360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:00:01.122360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:00:01.122360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:00:01.122360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:00:01.122360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 04:00:01.122360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 04:00:01.122360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 04:00:01.122360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 04:00:01.123329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 04:00:01.123329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 04:00:01.123329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 04:00:01.123329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 04:00:01.123329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:00:01.123329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:00:01.123329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:00:01.123329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:00:01.123329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:00:01.123329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:00:01.123329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:00:01.123329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:00:01.123329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:00:01.123329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:00:01.123329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:00:01.123329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:00:01.124205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:00:01.124205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:00:01.124205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:00:01.124205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:00:01.124205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 04:00:01.124205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 04:00:01.124205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 04:00:01.124205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 04:00:01.124205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 04:00:01.124205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 04:00:01.124205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:00:01.124205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:00:01.126232 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 541/1067] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 04:00:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:00:01.129767 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d2fe7-a031-43d6-b029-7349e97db761 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:01.234459 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d2fe7-a031-43d6-b029-7349e97db761 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:01.269496 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d914e1-92a0-431d-bf97-f1e499b7e348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:01.270159 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d914e1-92a0-431d-bf97-f1e499b7e348 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:01.270464 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d914e1-92a0-431d-bf97-f1e499b7e348 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:01.270743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d914e1-92a0-431d-bf97-f1e499b7e348 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:01.271493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d914e1-92a0-431d-bf97-f1e499b7e348 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:01.271802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d914e1-92a0-431d-bf97-f1e499b7e348 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:01.273171 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d914e1-92a0-431d-bf97-f1e499b7e348 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:01.287214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 527/1068] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 04:00:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:01.288500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7989d35-c883-4a07-9b1d-02dfa51101f9 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:01.403297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7989d35-c883-4a07-9b1d-02dfa51101f9 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:01.438781 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d532b1d-63f0-4dac-979c-6eb85bc050dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:01.439702 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d532b1d-63f0-4dac-979c-6eb85bc050dd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:01.439994 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d532b1d-63f0-4dac-979c-6eb85bc050dd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:01.440403 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d532b1d-63f0-4dac-979c-6eb85bc050dd None admin] PATH_INFO: `/v3/users/f5e303d4e85440178fb5e37a2ba6a2c0` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:01.443801 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:01.456514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d532b1d-63f0-4dac-979c-6eb85bc050dd None admin] RBAC: Authorizing `identity:get_user(user_id=f5e303d4e85440178fb5e37a2ba6a2c0)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:01.458036 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d532b1d-63f0-4dac-979c-6eb85bc050dd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:01.464534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 542/1069] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 04:00:01 2022] GET /identity/v3/users/f5e303d4e85440178fb5e37a2ba6a2c0 => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:01.465504 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d914e1-92a0-431d-bf97-f1e499b7e348 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:01.526382 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:01.570434 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d914e1-92a0-431d-bf97-f1e499b7e348 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:01.604979 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46cb35-c20f-4430-a284-5a1b4261577d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:01.605625 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46cb35-c20f-4430-a284-5a1b4261577d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:01.605913 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46cb35-c20f-4430-a284-5a1b4261577d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:01.606191 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46cb35-c20f-4430-a284-5a1b4261577d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:01.673814 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46cb35-c20f-4430-a284-5a1b4261577d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:01.675297 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46cb35-c20f-4430-a284-5a1b4261577d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:01.733725 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:01.782879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 528/1070] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 04:00:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 04:00:01.784366 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d532b1d-63f0-4dac-979c-6eb85bc050dd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:01.888930 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d532b1d-63f0-4dac-979c-6eb85bc050dd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:01.923808 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee01e0-995d-4ea8-a654-d2f5516284b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:01.924677 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee01e0-995d-4ea8-a654-d2f5516284b6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:01.924962 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee01e0-995d-4ea8-a654-d2f5516284b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:01.925240 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee01e0-995d-4ea8-a654-d2f5516284b6 None admin] PATH_INFO: `/v3/projects/45517a954fcb4bfe9bc89c2cf3ff487f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:01.926463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee01e0-995d-4ea8-a654-d2f5516284b6 None admin] RBAC: Authorizing `identity:get_project(project_id=45517a954fcb4bfe9bc89c2cf3ff487f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:01.927930 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee01e0-995d-4ea8-a654-d2f5516284b6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:01.934468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 543/1071] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 04:00:01 2022] GET /identity/v3/projects/45517a954fcb4bfe9bc89c2cf3ff487f => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:01.935981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46cb35-c20f-4430-a284-5a1b4261577d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:02.048843 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46cb35-c20f-4430-a284-5a1b4261577d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:02.089785 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc93e4-2b99-4777-b677-ce9daa1e3bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:02.090467 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc93e4-2b99-4777-b677-ce9daa1e3bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:02.090756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc93e4-2b99-4777-b677-ce9daa1e3bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:02.091033 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc93e4-2b99-4777-b677-ce9daa1e3bf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:02.091856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc93e4-2b99-4777-b677-ce9daa1e3bf5 None admin] RBAC: Adding query filter params (role.id=f0f7e28cbe314921afdfb657b163436f, scope.project.id=45517a954fcb4bfe9bc89c2cf3ff487f, user.id=f5e303d4e85440178fb5e37a2ba6a2c0) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:02.092171 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc93e4-2b99-4777-b677-ce9daa1e3bf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:02.093600 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc93e4-2b99-4777-b677-ce9daa1e3bf5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:02.104279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 529/1072] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 04:00:01 2022] GET /identity/v3/role_assignments?user.id=f5e303d4e85440178fb5e37a2ba6a2c0&scope.project.id=45517a954fcb4bfe9bc89c2cf3ff487f&role.id=f0f7e28cbe314921afdfb657b163436f => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:00:02.502692 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed0e7c-fc0d-4087-bb78-41a31db0c961 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:02.503044 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed0e7c-fc0d-4087-bb78-41a31db0c961 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:02.503269 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed0e7c-fc0d-4087-bb78-41a31db0c961 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:02.504789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 544/1073] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:00:02.510019 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cb786-ebc0-478e-8532-db49f7503373 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:02.510513 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cb786-ebc0-478e-8532-db49f7503373 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:02.510611 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972cb786-ebc0-478e-8532-db49f7503373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:02.527938 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-972cb786-ebc0-478e-8532-db49f7503373 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:02.529130 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-972cb786-ebc0-478e-8532-db49f7503373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:02.548464 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:02.602435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 530/1074] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:00:02.620863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed0e7c-fc0d-4087-bb78-41a31db0c961 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:02.684607 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:02.725410 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed0e7c-fc0d-4087-bb78-41a31db0c961 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:02.759527 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba672f1-1ff4-4eea-801e-ae28d00d67dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:02.760117 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba672f1-1ff4-4eea-801e-ae28d00d67dd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:02.760411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba672f1-1ff4-4eea-801e-ae28d00d67dd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:02.760692 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba672f1-1ff4-4eea-801e-ae28d00d67dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:02.830530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba672f1-1ff4-4eea-801e-ae28d00d67dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:02.831999 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba672f1-1ff4-4eea-801e-ae28d00d67dd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:02.863693 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f29fa6-854c-4f7e-b27f-995955061a7c None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:02.864078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f29fa6-854c-4f7e-b27f-995955061a7c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:02.864367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f29fa6-854c-4f7e-b27f-995955061a7c None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:02.865746 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 531/1075] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 04:00:02.871031 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9b05c-8b83-4994-a51a-05fea3dbffc5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:02.871318 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9b05c-8b83-4994-a51a-05fea3dbffc5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:02.871594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9b05c-8b83-4994-a51a-05fea3dbffc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:02.890627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3be9b05c-8b83-4994-a51a-05fea3dbffc5 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:02.893355 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3be9b05c-8b83-4994-a51a-05fea3dbffc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:02.897926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:02.938673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 545/1076] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 04:00:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 04:00:02.944401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba672f1-1ff4-4eea-801e-ae28d00d67dd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:02.966923 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 532/1077] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 04:00:02.990901 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9b05c-8b83-4994-a51a-05fea3dbffc5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:03.060387 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba672f1-1ff4-4eea-801e-ae28d00d67dd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:03.083000 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:03.097790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2b143-e61d-4017-9a0c-d09260762db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:03.098621 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2b143-e61d-4017-9a0c-d09260762db2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:03.098913 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2b143-e61d-4017-9a0c-d09260762db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:03.099194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2b143-e61d-4017-9a0c-d09260762db2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:03.099387 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9b05c-8b83-4994-a51a-05fea3dbffc5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:03.111064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2b143-e61d-4017-9a0c-d09260762db2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:03.112286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2b143-e61d-4017-9a0c-d09260762db2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:03.131195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-ecd2b143-e61d-4017-9a0c-d09260762db2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:00:03.131195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:00:03.131195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:00:03.131195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:00:03.131195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:00:03.131195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:00:03.131195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:00:03.131195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:00:03.131195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:00:03.131195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:00:03.131195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:00:03.131195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:00:03.131195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 04:00:03.131195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 04:00:03.131195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 04:00:03.132206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 04:00:03.132206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:00:03.132206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:00:03.132206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:00:03.132206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:00:03.132206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:00:03.132206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:00:03.132206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:00:03.132206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:00:03.132206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:00:03.132206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:00:03.132206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:00:03.132206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:00:03.132206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:00:03.132206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:00:03.132206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:00:03.132206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:00:03.133276 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 04:00:03.133276 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 04:00:03.133276 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:00:03.133276 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:00:03.133276 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 546/1078] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:00:03.138206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2b143-e61d-4017-9a0c-d09260762db2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:03.139077 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da394e3d-75ec-4de1-8580-84bef6aa1b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:03.139895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da394e3d-75ec-4de1-8580-84bef6aa1b6e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:03.140182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da394e3d-75ec-4de1-8580-84bef6aa1b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:03.140984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da394e3d-75ec-4de1-8580-84bef6aa1b6e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:03.151240 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da394e3d-75ec-4de1-8580-84bef6aa1b6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 04:00:03.154810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:03.160388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-da394e3d-75ec-4de1-8580-84bef6aa1b6e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 04:00:03.160388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:00:03.160388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:00:03.160388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:00:03.160388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:00:03.160388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:00:03.160388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:00:03.160388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:00:03.160388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:00:03.160388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:00:03.160388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:00:03.160388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:00:03.160388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 04:00:03.160388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 04:00:03.160388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 04:00:03.160388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 04:00:03.161332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 04:00:03.161332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 04:00:03.161332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 04:00:03.161332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 04:00:03.161332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:00:03.161332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:00:03.161332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:00:03.161332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:00:03.161332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:00:03.161332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:00:03.161332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:00:03.161332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:00:03.161332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:00:03.161332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:00:03.161332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:00:03.161332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:00:03.162190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:00:03.162190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:00:03.162190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:00:03.162190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:00:03.162190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 04:00:03.162190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 04:00:03.162190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 04:00:03.162190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 04:00:03.162190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 04:00:03.162190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 04:00:03.162190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 04:00:03.162190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:00:03.162190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 533/1079] 10.222.0.16 () {56 vars in 1120 bytes} [Fri Jun 24 04:00:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 04:00:03.167386 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da394e3d-75ec-4de1-8580-84bef6aa1b6e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:03.245906 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2b143-e61d-4017-9a0c-d09260762db2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:03.277489 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da394e3d-75ec-4de1-8580-84bef6aa1b6e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:03.283531 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251ee7d-7b80-4197-b722-8ceb1a197be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:03.284197 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251ee7d-7b80-4197-b722-8ceb1a197be3 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:03.284480 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251ee7d-7b80-4197-b722-8ceb1a197be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:03.284755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251ee7d-7b80-4197-b722-8ceb1a197be3 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:03.285477 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a251ee7d-7b80-4197-b722-8ceb1a197be3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:03.285783 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a251ee7d-7b80-4197-b722-8ceb1a197be3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:03.287346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a251ee7d-7b80-4197-b722-8ceb1a197be3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:03.299522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 547/1080] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 04:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:00:03.318075 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de5ad2-db0e-4a08-b40c-f99c2c951cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:03.319261 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de5ad2-db0e-4a08-b40c-f99c2c951cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:03.319767 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de5ad2-db0e-4a08-b40c-f99c2c951cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:03.320065 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de5ad2-db0e-4a08-b40c-f99c2c951cb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:03.321072 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de5ad2-db0e-4a08-b40c-f99c2c951cb4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:03.321624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de5ad2-db0e-4a08-b40c-f99c2c951cb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:03.323306 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de5ad2-db0e-4a08-b40c-f99c2c951cb4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:03.338358 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 534/1081] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Jun 24 04:00:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:00:03.339446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251ee7d-7b80-4197-b722-8ceb1a197be3 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:03.431396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:03.458946 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251ee7d-7b80-4197-b722-8ceb1a197be3 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:03.493491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af4330-c7ab-4751-ae49-c196b17bfd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:03.494287 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af4330-c7ab-4751-ae49-c196b17bfd8a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:03.494900 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af4330-c7ab-4751-ae49-c196b17bfd8a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:03.495186 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af4330-c7ab-4751-ae49-c196b17bfd8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:03.506592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af4330-c7ab-4751-ae49-c196b17bfd8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:03.508054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af4330-c7ab-4751-ae49-c196b17bfd8a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:03.525907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-02af4330-c7ab-4751-ae49-c196b17bfd8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:00:03.525907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:00:03.525907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:00:03.525907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:00:03.525907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:00:03.525907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:00:03.525907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:00:03.525907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:00:03.525907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:00:03.525907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:00:03.525907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:00:03.525907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:00:03.525907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 04:00:03.525907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 04:00:03.525907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 04:00:03.526938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 04:00:03.526938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:00:03.526938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:00:03.526938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:00:03.526938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:00:03.526938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:00:03.526938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:00:03.526938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:00:03.526938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:00:03.526938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:00:03.526938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:00:03.526938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:00:03.526938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:00:03.526938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:00:03.526938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:00:03.526938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:00:03.526938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:00:03.527879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 04:00:03.527879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 04:00:03.527879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:00:03.527879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:00:03.527879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 548/1082] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:00:03.534282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de5ad2-db0e-4a08-b40c-f99c2c951cb4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:03.548284 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:03.644840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de5ad2-db0e-4a08-b40c-f99c2c951cb4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:03.681887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36c2c3-32ba-48f4-ac16-bd8a70be2b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:03.682957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36c2c3-32ba-48f4-ac16-bd8a70be2b66 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:03.683545 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36c2c3-32ba-48f4-ac16-bd8a70be2b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:03.684012 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36c2c3-32ba-48f4-ac16-bd8a70be2b66 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:03.685217 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36c2c3-32ba-48f4-ac16-bd8a70be2b66 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:03.685711 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36c2c3-32ba-48f4-ac16-bd8a70be2b66 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:03.688546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36c2c3-32ba-48f4-ac16-bd8a70be2b66 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:03.703692 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 535/1083] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 04:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:03.709427 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af4330-c7ab-4751-ae49-c196b17bfd8a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:03.810933 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af4330-c7ab-4751-ae49-c196b17bfd8a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:03.836261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:03.844930 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb46787-217b-4f6f-bb2a-fd8077ced446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:03.845717 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46787-217b-4f6f-bb2a-fd8077ced446 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:03.852926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46787-217b-4f6f-bb2a-fd8077ced446 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:03.852926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46787-217b-4f6f-bb2a-fd8077ced446 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:03.853702 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb46787-217b-4f6f-bb2a-fd8077ced446 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:03.855193 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb46787-217b-4f6f-bb2a-fd8077ced446 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:03.869765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-adb46787-217b-4f6f-bb2a-fd8077ced446 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 04:00:03.869765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:00:03.869765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:00:03.869765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:00:03.869765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:00:03.869765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:00:03.869765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:00:03.869765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:00:03.869765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:00:03.869765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:00:03.869765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:00:03.869765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:00:03.869765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 04:00:03.869765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 04:00:03.869765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 04:00:03.869765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 04:00:03.870959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:00:03.870959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:00:03.870959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 04:00:03.870959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 04:00:03.870959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 04:00:03.870959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 04:00:03.870959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:00:03.870959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:00:03.870959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:00:03.870959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:00:03.870959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:00:03.870959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:00:03.870959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:00:03.870959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:00:03.870959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:00:03.870959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:00:03.870959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:00:03.870959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:00:03.871909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:00:03.871909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:00:03.871909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 04:00:03.871909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 04:00:03.871909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 04:00:03.871909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 04:00:03.871909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 04:00:03.871909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 04:00:03.871909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 04:00:03.871909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:00:03.871909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 549/1084] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Jun 24 04:00:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:00:03.877630 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36c2c3-32ba-48f4-ac16-bd8a70be2b66 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:03.942299 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:03.978736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36c2c3-32ba-48f4-ac16-bd8a70be2b66 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:04.016771 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447debaa-c13a-4619-a10c-dc6dd0e6dc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:04.017439 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447debaa-c13a-4619-a10c-dc6dd0e6dc53 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:04.017726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447debaa-c13a-4619-a10c-dc6dd0e6dc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:04.018008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447debaa-c13a-4619-a10c-dc6dd0e6dc53 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:04.018882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447debaa-c13a-4619-a10c-dc6dd0e6dc53 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:04.019198 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447debaa-c13a-4619-a10c-dc6dd0e6dc53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:04.020630 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447debaa-c13a-4619-a10c-dc6dd0e6dc53 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:04.043563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 536/1085] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Jun 24 04:00:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:04.044798 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb46787-217b-4f6f-bb2a-fd8077ced446 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:04.133791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab2a19-849e-461a-88ae-c2a028465d4b None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:04.134081 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab2a19-849e-461a-88ae-c2a028465d4b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:04.134397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab2a19-849e-461a-88ae-c2a028465d4b None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:04.135762 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 537/1086] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 04:00:04.140671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3178ac-036e-46d6-8c12-58d72f56bd16 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:04.141011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3178ac-036e-46d6-8c12-58d72f56bd16 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:04.141473 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3178ac-036e-46d6-8c12-58d72f56bd16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:04.142632 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb46787-217b-4f6f-bb2a-fd8077ced446 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:04.158862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-fe3178ac-036e-46d6-8c12-58d72f56bd16 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:04.160069 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fe3178ac-036e-46d6-8c12-58d72f56bd16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:04.176054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8ea4b-8ba0-4a16-b335-2f67e9967326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:04.176707 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8ea4b-8ba0-4a16-b335-2f67e9967326 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:04.176995 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8ea4b-8ba0-4a16-b335-2f67e9967326 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:04.177275 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8ea4b-8ba0-4a16-b335-2f67e9967326 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:04.227964 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 538/1087] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 04:00:04.231493 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:04.245557 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8ea4b-8ba0-4a16-b335-2f67e9967326 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:04.247051 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8ea4b-8ba0-4a16-b335-2f67e9967326 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:04.251688 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3178ac-036e-46d6-8c12-58d72f56bd16 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:04.305169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:04.344357 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 550/1088] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 04:00:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 04:00:04.349500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3178ac-036e-46d6-8c12-58d72f56bd16 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:04.350152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8ea4b-8ba0-4a16-b335-2f67e9967326 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:04.383743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909e0e2-8964-423c-9da3-111c9984f785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:04.384542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909e0e2-8964-423c-9da3-111c9984f785 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:04.384825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909e0e2-8964-423c-9da3-111c9984f785 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:04.385104 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909e0e2-8964-423c-9da3-111c9984f785 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:04.395430 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4909e0e2-8964-423c-9da3-111c9984f785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 04:00:04.403144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-4909e0e2-8964-423c-9da3-111c9984f785 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:00:04.403144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:00:04.403144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:00:04.403144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:00:04.403144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:00:04.403144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:00:04.403144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:00:04.403144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:00:04.403144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:00:04.403144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:00:04.403144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:00:04.403144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:00:04.403144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 04:00:04.403144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 04:00:04.403144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 04:00:04.403144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 04:00:04.404094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 04:00:04.404094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 04:00:04.404094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 04:00:04.404094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 04:00:04.404094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:00:04.404094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:00:04.404094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:00:04.404094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:00:04.404094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:00:04.404094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:00:04.404094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:00:04.404094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:00:04.404094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:00:04.404094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:00:04.404094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:00:04.404094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:00:04.404942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:00:04.404942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:00:04.404942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:00:04.404942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:00:04.404942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 04:00:04.404942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 04:00:04.404942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 04:00:04.404942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 04:00:04.404942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 04:00:04.404942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 04:00:04.404942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:00:04.404942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:00:04.404942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 539/1089] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 04:00:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:00:04.411038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909e0e2-8964-423c-9da3-111c9984f785 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:04.432770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:04.447660 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8ea4b-8ba0-4a16-b335-2f67e9967326 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:04.480732 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a6acc-cfd1-4fbd-a716-212f24588eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:04.481442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a6acc-cfd1-4fbd-a716-212f24588eaa None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:04.481725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a6acc-cfd1-4fbd-a716-212f24588eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:04.482001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a6acc-cfd1-4fbd-a716-212f24588eaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:04.492834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a6acc-cfd1-4fbd-a716-212f24588eaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:04.494271 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a6acc-cfd1-4fbd-a716-212f24588eaa None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:04.511897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909e0e2-8964-423c-9da3-111c9984f785 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:04.512322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-e12a6acc-cfd1-4fbd-a716-212f24588eaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:00:04.512322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:00:04.512322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:00:04.512322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:00:04.512322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:00:04.512322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:00:04.512322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:00:04.512322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:00:04.512322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:00:04.512322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:00:04.512322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:00:04.512322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:00:04.512322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 04:00:04.512322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 04:00:04.512322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 04:00:04.513265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 04:00:04.513265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:00:04.513265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:00:04.513265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:00:04.513265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:00:04.513265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:00:04.513265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:00:04.513265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:00:04.513265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:00:04.513265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:00:04.513265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:00:04.513265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:00:04.513265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:00:04.513265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:00:04.513265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:00:04.513265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:00:04.513265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:00:04.514171 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 04:00:04.514171 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 04:00:04.514171 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:00:04.514171 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:00:04.514171 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 551/1090] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:00:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 04:00:04.519704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a6acc-cfd1-4fbd-a716-212f24588eaa None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:04.526899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:04.546150 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b6699-9f87-43ec-8535-d678a4e2acc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:04.546844 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b6699-9f87-43ec-8535-d678a4e2acc3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:04.547868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b6699-9f87-43ec-8535-d678a4e2acc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:04.548146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b6699-9f87-43ec-8535-d678a4e2acc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:04.548894 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b6699-9f87-43ec-8535-d678a4e2acc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:04.549207 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b6699-9f87-43ec-8535-d678a4e2acc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:04.550605 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52b6699-9f87-43ec-8535-d678a4e2acc3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:04.564307 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 540/1091] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 04:00:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:00:04.565352 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b6699-9f87-43ec-8535-d678a4e2acc3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:04.617715 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a6acc-cfd1-4fbd-a716-212f24588eaa None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:04.651165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e767d84-2c9c-4356-8d96-9a1ae81b54cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:04.651824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e767d84-2c9c-4356-8d96-9a1ae81b54cf None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:04.652108 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e767d84-2c9c-4356-8d96-9a1ae81b54cf None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:04.652385 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e767d84-2c9c-4356-8d96-9a1ae81b54cf None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:04.653107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e767d84-2c9c-4356-8d96-9a1ae81b54cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:04.653417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e767d84-2c9c-4356-8d96-9a1ae81b54cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:04.654835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e767d84-2c9c-4356-8d96-9a1ae81b54cf None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:04.662295 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:04.665400 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b6699-9f87-43ec-8535-d678a4e2acc3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:04.667271 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 552/1092] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 04:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:00:04.673739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e767d84-2c9c-4356-8d96-9a1ae81b54cf None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:04.700321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0eb2d-2ac9-42c7-b71f-b8c613cd95ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:04.701420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0eb2d-2ac9-42c7-b71f-b8c613cd95ac None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:04.701707 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0eb2d-2ac9-42c7-b71f-b8c613cd95ac None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:04.701990 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0eb2d-2ac9-42c7-b71f-b8c613cd95ac None admin] PATH_INFO: `/v3/users/f5e303d4e85440178fb5e37a2ba6a2c0` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:04.703427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0eb2d-2ac9-42c7-b71f-b8c613cd95ac None admin] RBAC: Authorizing `identity:get_user(user_id=f5e303d4e85440178fb5e37a2ba6a2c0)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:04.705266 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0eb2d-2ac9-42c7-b71f-b8c613cd95ac None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:04.706928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 541/1093] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 04:00:04 2022] GET /identity/v3/users/f5e303d4e85440178fb5e37a2ba6a2c0 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:00:04.712399 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0eb2d-2ac9-42c7-b71f-b8c613cd95ac None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:04.760259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:04.774917 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e767d84-2c9c-4356-8d96-9a1ae81b54cf None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:04.808303 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934005d2-b0a6-415a-840a-4479dff21bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:04.809086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934005d2-b0a6-415a-840a-4479dff21bab None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:04.809368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934005d2-b0a6-415a-840a-4479dff21bab None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:04.809644 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934005d2-b0a6-415a-840a-4479dff21bab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:04.813440 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0eb2d-2ac9-42c7-b71f-b8c613cd95ac None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:04.816263 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934005d2-b0a6-415a-840a-4479dff21bab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:04.817691 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934005d2-b0a6-415a-840a-4479dff21bab None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:04.831850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-934005d2-b0a6-415a-840a-4479dff21bab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 04:00:04.831850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:00:04.831850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:00:04.831850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:00:04.831850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:00:04.831850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:00:04.831850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:00:04.831850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:00:04.831850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:00:04.831850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:00:04.831850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:00:04.831850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:00:04.831850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 04:00:04.831850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 04:00:04.831850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 04:00:04.832752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 04:00:04.832752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:00:04.832752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:00:04.832752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:00:04.832752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:00:04.832752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:00:04.832752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:00:04.832752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:00:04.832752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:00:04.832752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:00:04.832752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:00:04.832752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:00:04.832752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:00:04.832752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:00:04.832752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:00:04.832752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:00:04.832752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:00:04.833680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 04:00:04.833680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 04:00:04.833680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 04:00:04.833680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 04:00:04.833680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 04:00:04.833680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 04:00:04.833680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 04:00:04.833680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:00:04.833680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 553/1094] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 04:00:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 04:00:04.838817 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934005d2-b0a6-415a-840a-4479dff21bab None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:04.848205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776adf02-7bae-48c8-ab20-d2d5d003fe43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:04.849069 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776adf02-7bae-48c8-ab20-d2d5d003fe43 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:04.849354 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776adf02-7bae-48c8-ab20-d2d5d003fe43 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:04.849633 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776adf02-7bae-48c8-ab20-d2d5d003fe43 None admin] PATH_INFO: `/v3/projects/45517a954fcb4bfe9bc89c2cf3ff487f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:04.850838 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776adf02-7bae-48c8-ab20-d2d5d003fe43 None admin] RBAC: Authorizing `identity:get_project(project_id=45517a954fcb4bfe9bc89c2cf3ff487f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:04.852269 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776adf02-7bae-48c8-ab20-d2d5d003fe43 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:04.853964 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 542/1095] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 04:00:04 2022] GET /identity/v3/projects/45517a954fcb4bfe9bc89c2cf3ff487f => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:00:04.860238 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776adf02-7bae-48c8-ab20-d2d5d003fe43 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:04.897818 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:04.939442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934005d2-b0a6-415a-840a-4479dff21bab None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:04.961175 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776adf02-7bae-48c8-ab20-d2d5d003fe43 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:04.972945 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5468a5-4546-4a89-99d1-bc0c47f0c3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:04.973606 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5468a5-4546-4a89-99d1-bc0c47f0c3bb None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:04.973892 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5468a5-4546-4a89-99d1-bc0c47f0c3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:04.974167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5468a5-4546-4a89-99d1-bc0c47f0c3bb None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:04.974995 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5468a5-4546-4a89-99d1-bc0c47f0c3bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:04.975309 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:04.975424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5468a5-4546-4a89-99d1-bc0c47f0c3bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:04.976742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5468a5-4546-4a89-99d1-bc0c47f0c3bb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:04.989763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 554/1096] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 04:00:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:00:04.996258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5468a5-4546-4a89-99d1-bc0c47f0c3bb None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:04.996657 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680941f-1604-4066-8935-cc17a33eba48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:04.997550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680941f-1604-4066-8935-cc17a33eba48 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:04.997829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680941f-1604-4066-8935-cc17a33eba48 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:04.998103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680941f-1604-4066-8935-cc17a33eba48 None admin] PATH_INFO: `/v3/projects/45517a954fcb4bfe9bc89c2cf3ff487f/users/f5e303d4e85440178fb5e37a2ba6a2c0/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:05.000301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680941f-1604-4066-8935-cc17a33eba48 None admin] RBAC: Authorizing `identity:create_grant(project_id=45517a954fcb4bfe9bc89c2cf3ff487f, user_id=f5e303d4e85440178fb5e37a2ba6a2c0, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:05.001730 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680941f-1604-4066-8935-cc17a33eba48 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:05.095996 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5468a5-4546-4a89-99d1-bc0c47f0c3bb None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:05.105694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 543/1097] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 04:00:04 2022] PUT /identity/v3/projects/45517a954fcb4bfe9bc89c2cf3ff487f/users/f5e303d4e85440178fb5e37a2ba6a2c0/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:00:05.122531 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:05.131557 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5330bd9-0e8c-4e6f-b211-5866000f070f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:05.132244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5330bd9-0e8c-4e6f-b211-5866000f070f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:05.132532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5330bd9-0e8c-4e6f-b211-5866000f070f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:05.132812 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5330bd9-0e8c-4e6f-b211-5866000f070f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:05.133681 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5330bd9-0e8c-4e6f-b211-5866000f070f None admin] RBAC: Adding query filter params (role.id=254f32a34d7b4de7b5c039848b8153d6, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=80b7b3ec60db4de19bbf7d08f111f470) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:05.133994 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5330bd9-0e8c-4e6f-b211-5866000f070f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:05.135516 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5330bd9-0e8c-4e6f-b211-5866000f070f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:05.142804 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 555/1098] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 04:00:04 2022] GET /identity/v3/role_assignments?user.id=80b7b3ec60db4de19bbf7d08f111f470&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=254f32a34d7b4de7b5c039848b8153d6 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:00:07.168909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6c5a2-ff5b-4848-868f-7d073f5bc077 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:07.169457 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6c5a2-ff5b-4848-868f-7d073f5bc077 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:07.169457 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6c5a2-ff5b-4848-868f-7d073f5bc077 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:07.170887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 544/1099] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:00:07.176642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b130d06-b97e-41b1-b659-5930e32dc3f6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:07.177045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b130d06-b97e-41b1-b659-5930e32dc3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:07.177199 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b130d06-b97e-41b1-b659-5930e32dc3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:07.193783 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1b130d06-b97e-41b1-b659-5930e32dc3f6 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:07.195005 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1b130d06-b97e-41b1-b659-5930e32dc3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:07.259232 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 556/1100] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:00:07.283993 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6c5a2-ff5b-4848-868f-7d073f5bc077 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:07.382759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6c5a2-ff5b-4848-868f-7d073f5bc077 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:07.420623 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6905c-63c5-47fb-8cb0-79758c24c566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:07.421405 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6905c-63c5-47fb-8cb0-79758c24c566 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:07.421687 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6905c-63c5-47fb-8cb0-79758c24c566 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:07.422035 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6905c-63c5-47fb-8cb0-79758c24c566 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:07.436042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6905c-63c5-47fb-8cb0-79758c24c566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 04:00:07.444837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-61a6905c-63c5-47fb-8cb0-79758c24c566 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:00:07.444837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:00:07.444837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:00:07.444837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:00:07.444837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:00:07.444837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:00:07.444837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:00:07.444837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:00:07.444837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:00:07.444837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:00:07.444837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:00:07.444837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:00:07.444837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 04:00:07.444837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 04:00:07.444837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 04:00:07.444837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 04:00:07.446034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 04:00:07.446034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 04:00:07.446034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 04:00:07.446034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 04:00:07.446034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:00:07.446034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:00:07.446034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:00:07.446034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:00:07.446034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:00:07.446034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:00:07.446034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:00:07.446034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:00:07.446034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:00:07.446034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:00:07.446034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:00:07.446034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:00:07.451778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:00:07.451778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:00:07.451778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:00:07.451778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:00:07.451778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 04:00:07.451778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 04:00:07.451778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 04:00:07.451778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 04:00:07.451778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 04:00:07.451778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 04:00:07.451778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:00:07.451778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:00:07.451778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 545/1101] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 04:00:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:00:07.452484 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b130d06-b97e-41b1-b659-5930e32dc3f6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:07.548540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b130d06-b97e-41b1-b659-5930e32dc3f6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:07.560960 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:07.581569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236cf6d-f4bb-4f36-8714-81b94fab0eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:07.582233 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236cf6d-f4bb-4f36-8714-81b94fab0eae None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:07.589918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236cf6d-f4bb-4f36-8714-81b94fab0eae None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:07.589918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236cf6d-f4bb-4f36-8714-81b94fab0eae None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:07.589918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236cf6d-f4bb-4f36-8714-81b94fab0eae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:07.589918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236cf6d-f4bb-4f36-8714-81b94fab0eae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:07.589918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236cf6d-f4bb-4f36-8714-81b94fab0eae None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:07.593174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 557/1102] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 04:00:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:07.599729 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6905c-63c5-47fb-8cb0-79758c24c566 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:07.631313 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:07.697917 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6905c-63c5-47fb-8cb0-79758c24c566 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:07.730440 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bb6e0-1a3c-4b41-87c3-551e647d6a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:07.731300 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bb6e0-1a3c-4b41-87c3-551e647d6a50 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:07.731579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bb6e0-1a3c-4b41-87c3-551e647d6a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:07.731854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bb6e0-1a3c-4b41-87c3-551e647d6a50 None admin] PATH_INFO: `/v3/users/f5e303d4e85440178fb5e37a2ba6a2c0` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:07.733032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6bb6e0-1a3c-4b41-87c3-551e647d6a50 None admin] RBAC: Authorizing `identity:get_user(user_id=f5e303d4e85440178fb5e37a2ba6a2c0)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:07.734543 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6bb6e0-1a3c-4b41-87c3-551e647d6a50 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:07.736162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 546/1103] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 04:00:07 2022] GET /identity/v3/users/f5e303d4e85440178fb5e37a2ba6a2c0 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:07.741980 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236cf6d-f4bb-4f36-8714-81b94fab0eae None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:07.838956 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236cf6d-f4bb-4f36-8714-81b94fab0eae None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:07.870782 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289acce5-12f0-4edc-9a4b-f6cb250df3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:07.871420 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289acce5-12f0-4edc-9a4b-f6cb250df3de None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:07.871708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289acce5-12f0-4edc-9a4b-f6cb250df3de None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:07.871986 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289acce5-12f0-4edc-9a4b-f6cb250df3de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:07.913294 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:07.937016 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289acce5-12f0-4edc-9a4b-f6cb250df3de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:07.939106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289acce5-12f0-4edc-9a4b-f6cb250df3de None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:08.035557 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 558/1104] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 04:00:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 04:00:08.041343 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bb6e0-1a3c-4b41-87c3-551e647d6a50 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:08.138524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:08.141451 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bb6e0-1a3c-4b41-87c3-551e647d6a50 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:08.174242 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16b5e2-715c-4bab-b20e-49e19434a4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:08.175119 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16b5e2-715c-4bab-b20e-49e19434a4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:08.175405 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16b5e2-715c-4bab-b20e-49e19434a4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:08.175684 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16b5e2-715c-4bab-b20e-49e19434a4a0 None admin] PATH_INFO: `/v3/projects/45517a954fcb4bfe9bc89c2cf3ff487f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:08.176867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f16b5e2-715c-4bab-b20e-49e19434a4a0 None admin] RBAC: Authorizing `identity:get_project(project_id=45517a954fcb4bfe9bc89c2cf3ff487f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:08.178312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f16b5e2-715c-4bab-b20e-49e19434a4a0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:08.180034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 547/1105] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 04:00:08 2022] GET /identity/v3/projects/45517a954fcb4bfe9bc89c2cf3ff487f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:08.186253 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289acce5-12f0-4edc-9a4b-f6cb250df3de None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:08.258898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:08.282467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289acce5-12f0-4edc-9a4b-f6cb250df3de None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:08.315719 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5396cf-b8c0-4440-b9af-1f9f7f6d1a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:08.316309 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5396cf-b8c0-4440-b9af-1f9f7f6d1a07 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:08.316595 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5396cf-b8c0-4440-b9af-1f9f7f6d1a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:08.316873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5396cf-b8c0-4440-b9af-1f9f7f6d1a07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:08.317701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5396cf-b8c0-4440-b9af-1f9f7f6d1a07 None admin] RBAC: Adding query filter params (role.id=f0f7e28cbe314921afdfb657b163436f, scope.project.id=45517a954fcb4bfe9bc89c2cf3ff487f, user.id=f5e303d4e85440178fb5e37a2ba6a2c0) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:08.318014 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5396cf-b8c0-4440-b9af-1f9f7f6d1a07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:08.319473 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5396cf-b8c0-4440-b9af-1f9f7f6d1a07 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:08.325131 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 559/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 04:00:08 2022] GET /identity/v3/role_assignments?user.id=f5e303d4e85440178fb5e37a2ba6a2c0&scope.project.id=45517a954fcb4bfe9bc89c2cf3ff487f&role.id=f0f7e28cbe314921afdfb657b163436f => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:10.387442 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf10911-9104-4f1f-9d26-644187df4e56 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:10.388004 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf10911-9104-4f1f-9d26-644187df4e56 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:10.388004 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf10911-9104-4f1f-9d26-644187df4e56 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:10.389411 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 548/1107] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:00:10.395835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77799f6-18f0-44df-a195-86c10199e71c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:10.396341 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77799f6-18f0-44df-a195-86c10199e71c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:10.396429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77799f6-18f0-44df-a195-86c10199e71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:10.414085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e77799f6-18f0-44df-a195-86c10199e71c None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:10.415316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e77799f6-18f0-44df-a195-86c10199e71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:10.481320 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 560/1108] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:00:10.507181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf10911-9104-4f1f-9d26-644187df4e56 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:10.606079 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf10911-9104-4f1f-9d26-644187df4e56 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:10.638966 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02944014-7cd2-4041-9345-37604df2123c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:10.639677 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02944014-7cd2-4041-9345-37604df2123c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:10.639959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02944014-7cd2-4041-9345-37604df2123c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:10.640232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02944014-7cd2-4041-9345-37604df2123c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:10.641924 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02944014-7cd2-4041-9345-37604df2123c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:10.643377 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02944014-7cd2-4041-9345-37604df2123c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:10.644959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 549/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:00:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:10.650720 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77799f6-18f0-44df-a195-86c10199e71c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:10.716091 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:10.751171 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77799f6-18f0-44df-a195-86c10199e71c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:10.784377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4fae8-edf7-4c69-8689-1577c8ac1c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:10.785050 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4fae8-edf7-4c69-8689-1577c8ac1c60 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:10.785337 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4fae8-edf7-4c69-8689-1577c8ac1c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:10.785615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4fae8-edf7-4c69-8689-1577c8ac1c60 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:10.786326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4fae8-edf7-4c69-8689-1577c8ac1c60 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:10.787799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4fae8-edf7-4c69-8689-1577c8ac1c60 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:10.888401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 561/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jun 24 04:00:10 2022] POST /identity/v3/users => generated 286 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:00:13.002033 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710761b-3d69-4a42-b8dc-040ed49faf5c None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:13.002600 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710761b-3d69-4a42-b8dc-040ed49faf5c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:13.002600 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5710761b-3d69-4a42-b8dc-040ed49faf5c None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:13.004000 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 550/1111] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:00:13.009241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba05b46-23fc-4745-854b-464d16d47cb2 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:13.009685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba05b46-23fc-4745-854b-464d16d47cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:13.009805 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba05b46-23fc-4745-854b-464d16d47cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:13.027041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1ba05b46-23fc-4745-854b-464d16d47cb2 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:13.028226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1ba05b46-23fc-4745-854b-464d16d47cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:13.093569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 562/1112] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:00:13.117997 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5710761b-3d69-4a42-b8dc-040ed49faf5c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:13.154191 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:13.217704 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5710761b-3d69-4a42-b8dc-040ed49faf5c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:13.250282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc1948-4d75-4bb5-93c4-c918e13b7e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:13.251152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc1948-4d75-4bb5-93c4-c918e13b7e98 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:13.251433 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc1948-4d75-4bb5-93c4-c918e13b7e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:13.251709 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc1948-4d75-4bb5-93c4-c918e13b7e98 None admin] PATH_INFO: `/v3/roles/a048041c27f44edf98a110d23138a327` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:13.253228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc1948-4d75-4bb5-93c4-c918e13b7e98 None admin] RBAC: Authorizing `identity:get_role(role_id=a048041c27f44edf98a110d23138a327)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:13.254566 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc1948-4d75-4bb5-93c4-c918e13b7e98 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:13.255922 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 551/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 04:00:13 2022] GET /identity/v3/roles/a048041c27f44edf98a110d23138a327 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:13.261533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba05b46-23fc-4745-854b-464d16d47cb2 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:13.315246 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:13.356992 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba05b46-23fc-4745-854b-464d16d47cb2 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:13.390916 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af507fd-c11c-42a0-9211-730b2f92ae5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:13.391912 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af507fd-c11c-42a0-9211-730b2f92ae5c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:13.392197 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af507fd-c11c-42a0-9211-730b2f92ae5c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:13.392474 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af507fd-c11c-42a0-9211-730b2f92ae5c None admin] PATH_INFO: `/v3/users/37ee96a2cbc649aea5fff5cd56ed35f6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:13.405038 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af507fd-c11c-42a0-9211-730b2f92ae5c None admin] RBAC: Authorizing `identity:get_user(user_id=37ee96a2cbc649aea5fff5cd56ed35f6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:13.406572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af507fd-c11c-42a0-9211-730b2f92ae5c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:13.408238 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 563/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 04:00:13 2022] GET /identity/v3/users/37ee96a2cbc649aea5fff5cd56ed35f6 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:13.413902 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc1948-4d75-4bb5-93c4-c918e13b7e98 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:13.516838 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc1948-4d75-4bb5-93c4-c918e13b7e98 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:13.519041 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:13.550089 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd989cd-a9d2-4539-b875-0bde14fdc11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:13.550752 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd989cd-a9d2-4539-b875-0bde14fdc11e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:13.551033 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd989cd-a9d2-4539-b875-0bde14fdc11e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:13.551307 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd989cd-a9d2-4539-b875-0bde14fdc11e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:13.616538 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd989cd-a9d2-4539-b875-0bde14fdc11e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:13.617973 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd989cd-a9d2-4539-b875-0bde14fdc11e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:13.711424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 552/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 04:00:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 04:00:13.717356 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af507fd-c11c-42a0-9211-730b2f92ae5c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:13.812927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af507fd-c11c-42a0-9211-730b2f92ae5c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:13.825336 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:13.846158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dced15-4a5b-47b0-92b7-e1caf9f7988d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:13.847020 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dced15-4a5b-47b0-92b7-e1caf9f7988d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:13.850465 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dced15-4a5b-47b0-92b7-e1caf9f7988d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:13.850465 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dced15-4a5b-47b0-92b7-e1caf9f7988d None admin] PATH_INFO: `/v3/projects/45517a954fcb4bfe9bc89c2cf3ff487f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:13.850465 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dced15-4a5b-47b0-92b7-e1caf9f7988d None admin] RBAC: Authorizing `identity:get_project(project_id=45517a954fcb4bfe9bc89c2cf3ff487f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:13.850465 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dced15-4a5b-47b0-92b7-e1caf9f7988d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:13.851886 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 564/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 04:00:13 2022] GET /identity/v3/projects/45517a954fcb4bfe9bc89c2cf3ff487f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:13.857785 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd989cd-a9d2-4539-b875-0bde14fdc11e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:13.861502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:13.954816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd989cd-a9d2-4539-b875-0bde14fdc11e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:13.987073 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6bb08-ece6-43f0-aad3-aef6c57d1b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:13.987713 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6bb08-ece6-43f0-aad3-aef6c57d1b23 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:13.987996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6bb08-ece6-43f0-aad3-aef6c57d1b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:13.988272 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6bb08-ece6-43f0-aad3-aef6c57d1b23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:13.989095 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6bb08-ece6-43f0-aad3-aef6c57d1b23 None admin] RBAC: Adding query filter params (role.id=a048041c27f44edf98a110d23138a327, scope.project.id=45517a954fcb4bfe9bc89c2cf3ff487f, user.id=37ee96a2cbc649aea5fff5cd56ed35f6) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:13.989409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6bb08-ece6-43f0-aad3-aef6c57d1b23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:13.990869 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6bb08-ece6-43f0-aad3-aef6c57d1b23 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:13.996429 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 553/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 04:00:13 2022] GET /identity/v3/role_assignments?user.id=37ee96a2cbc649aea5fff5cd56ed35f6&scope.project.id=45517a954fcb4bfe9bc89c2cf3ff487f&role.id=a048041c27f44edf98a110d23138a327 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:15.986892 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac8763-6929-46f9-a2d9-a303b69c7fc7 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:15.987430 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac8763-6929-46f9-a2d9-a303b69c7fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:15.987430 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac8763-6929-46f9-a2d9-a303b69c7fc7 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:15.988814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 565/1118] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:00:15.994093 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808df085-2119-4726-a666-f367540310d2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:15.994782 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808df085-2119-4726-a666-f367540310d2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:15.994929 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808df085-2119-4726-a666-f367540310d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:16.011199 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-808df085-2119-4726-a666-f367540310d2 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:16.012374 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-808df085-2119-4726-a666-f367540310d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:16.073235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:16.078707 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 554/1119] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:00:16.102284 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac8763-6929-46f9-a2d9-a303b69c7fc7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:16.200109 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac8763-6929-46f9-a2d9-a303b69c7fc7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:16.232585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a7ec2-ff85-40c6-9daf-b23651ff7cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:16.233427 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a7ec2-ff85-40c6-9daf-b23651ff7cab None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:16.233711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a7ec2-ff85-40c6-9daf-b23651ff7cab None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:16.234044 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a7ec2-ff85-40c6-9daf-b23651ff7cab None admin] PATH_INFO: `/v3/roles/a048041c27f44edf98a110d23138a327` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:16.235793 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a7ec2-ff85-40c6-9daf-b23651ff7cab None admin] RBAC: Authorizing `identity:get_role(role_id=a048041c27f44edf98a110d23138a327)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:16.237211 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a7ec2-ff85-40c6-9daf-b23651ff7cab None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:16.238538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 566/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 04:00:16 2022] GET /identity/v3/roles/a048041c27f44edf98a110d23138a327 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:16.244799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808df085-2119-4726-a666-f367540310d2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:16.341605 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808df085-2119-4726-a666-f367540310d2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:16.375943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb00396-7eea-4e3c-ba4e-da3c6d09e67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:16.376796 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb00396-7eea-4e3c-ba4e-da3c6d09e67d None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:16.377079 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb00396-7eea-4e3c-ba4e-da3c6d09e67d None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:16.377353 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb00396-7eea-4e3c-ba4e-da3c6d09e67d None admin] PATH_INFO: `/v3/users/37ee96a2cbc649aea5fff5cd56ed35f6` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:16.378576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb00396-7eea-4e3c-ba4e-da3c6d09e67d None admin] RBAC: Authorizing `identity:get_user(user_id=37ee96a2cbc649aea5fff5cd56ed35f6)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:16.380007 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb00396-7eea-4e3c-ba4e-da3c6d09e67d None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:16.381618 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 555/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 04:00:16 2022] GET /identity/v3/users/37ee96a2cbc649aea5fff5cd56ed35f6 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:16.387344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a7ec2-ff85-40c6-9daf-b23651ff7cab None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:16.435299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:16.487527 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a7ec2-ff85-40c6-9daf-b23651ff7cab None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:16.520425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd3688-b42f-450c-9995-b90d9972a947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:16.521278 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd3688-b42f-450c-9995-b90d9972a947 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:16.521564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd3688-b42f-450c-9995-b90d9972a947 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:16.521842 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd3688-b42f-450c-9995-b90d9972a947 None admin] PATH_INFO: `/v3/projects/45517a954fcb4bfe9bc89c2cf3ff487f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:16.523032 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd3688-b42f-450c-9995-b90d9972a947 None admin] RBAC: Authorizing `identity:get_project(project_id=45517a954fcb4bfe9bc89c2cf3ff487f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:16.524464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd3688-b42f-450c-9995-b90d9972a947 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:16.526151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 567/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 04:00:16 2022] GET /identity/v3/projects/45517a954fcb4bfe9bc89c2cf3ff487f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:16.532508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb00396-7eea-4e3c-ba4e-da3c6d09e67d None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:16.595923 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:16.629620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb00396-7eea-4e3c-ba4e-da3c6d09e67d None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:16.662445 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bb96f7-b33b-42f6-a66f-bf33028e33fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:16.663323 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb96f7-b33b-42f6-a66f-bf33028e33fd None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:16.663609 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb96f7-b33b-42f6-a66f-bf33028e33fd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:16.663892 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bb96f7-b33b-42f6-a66f-bf33028e33fd None admin] PATH_INFO: `/v3/projects/45517a954fcb4bfe9bc89c2cf3ff487f/users/37ee96a2cbc649aea5fff5cd56ed35f6/roles/a048041c27f44edf98a110d23138a327` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:16.666089 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb96f7-b33b-42f6-a66f-bf33028e33fd None admin] RBAC: Authorizing `identity:create_grant(project_id=45517a954fcb4bfe9bc89c2cf3ff487f, user_id=37ee96a2cbc649aea5fff5cd56ed35f6, role_id=a048041c27f44edf98a110d23138a327)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:16.667565 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bb96f7-b33b-42f6-a66f-bf33028e33fd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:16.727722 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 556/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 04:00:16 2022] PUT /identity/v3/projects/45517a954fcb4bfe9bc89c2cf3ff487f/users/37ee96a2cbc649aea5fff5cd56ed35f6/roles/a048041c27f44edf98a110d23138a327 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:00:18.762241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16f187-6f25-4c99-a849-e874aa75ae1e None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:18.762938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16f187-6f25-4c99-a849-e874aa75ae1e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:18.762938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a16f187-6f25-4c99-a849-e874aa75ae1e None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:18.764212 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 568/1124] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:00:18.770128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44f769-62f6-40aa-8e2b-f128774f1280 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:18.770577 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44f769-62f6-40aa-8e2b-f128774f1280 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:18.770716 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44f769-62f6-40aa-8e2b-f128774f1280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:18.788284 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2a44f769-62f6-40aa-8e2b-f128774f1280 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:18.789467 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2a44f769-62f6-40aa-8e2b-f128774f1280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:18.854264 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 557/1125] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:00:18.879053 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16f187-6f25-4c99-a849-e874aa75ae1e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:18.977574 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a16f187-6f25-4c99-a849-e874aa75ae1e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:18.993433 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:19.012055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6977134-3285-4f5a-9f03-c12aabe4c89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:19.012955 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6977134-3285-4f5a-9f03-c12aabe4c89a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:19.013192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6977134-3285-4f5a-9f03-c12aabe4c89a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:19.013469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6977134-3285-4f5a-9f03-c12aabe4c89a None admin] PATH_INFO: `/v3/roles/a048041c27f44edf98a110d23138a327` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:19.014976 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6977134-3285-4f5a-9f03-c12aabe4c89a None admin] RBAC: Authorizing `identity:get_role(role_id=a048041c27f44edf98a110d23138a327)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:19.016916 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6977134-3285-4f5a-9f03-c12aabe4c89a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:19.018262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 569/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 04:00:18 2022] GET /identity/v3/roles/a048041c27f44edf98a110d23138a327 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:19.024064 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44f769-62f6-40aa-8e2b-f128774f1280 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:19.088059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:19.130703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44f769-62f6-40aa-8e2b-f128774f1280 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:19.164246 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8f31d-daed-449d-87ef-2a026b441d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:19.165121 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8f31d-daed-449d-87ef-2a026b441d49 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:19.165413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8f31d-daed-449d-87ef-2a026b441d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:19.165691 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8f31d-daed-449d-87ef-2a026b441d49 None admin] PATH_INFO: `/v3/users/37ee96a2cbc649aea5fff5cd56ed35f6` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:19.166917 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8f31d-daed-449d-87ef-2a026b441d49 None admin] RBAC: Authorizing `identity:get_user(user_id=37ee96a2cbc649aea5fff5cd56ed35f6)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:19.168367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8f31d-daed-449d-87ef-2a026b441d49 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:19.170014 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 558/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 04:00:19 2022] GET /identity/v3/users/37ee96a2cbc649aea5fff5cd56ed35f6 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:19.175773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6977134-3285-4f5a-9f03-c12aabe4c89a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:19.272191 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6977134-3285-4f5a-9f03-c12aabe4c89a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:19.304793 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f678b1-8c53-43ac-950b-28f67b800df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:19.305430 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f678b1-8c53-43ac-950b-28f67b800df4 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:19.305719 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f678b1-8c53-43ac-950b-28f67b800df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:19.305998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f678b1-8c53-43ac-950b-28f67b800df4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:19.357492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:19.373773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f678b1-8c53-43ac-950b-28f67b800df4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:19.375855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f678b1-8c53-43ac-950b-28f67b800df4 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:19.471227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 570/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 04:00:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 04:00:19.479379 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8f31d-daed-449d-87ef-2a026b441d49 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:19.589359 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8f31d-daed-449d-87ef-2a026b441d49 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:19.627028 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:19.640520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f054c33f-e5cd-4f65-8927-1ddd81646d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:19.642088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054c33f-e5cd-4f65-8927-1ddd81646d17 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:19.642640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054c33f-e5cd-4f65-8927-1ddd81646d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:19.643091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f054c33f-e5cd-4f65-8927-1ddd81646d17 None admin] PATH_INFO: `/v3/projects/45517a954fcb4bfe9bc89c2cf3ff487f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:19.645212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054c33f-e5cd-4f65-8927-1ddd81646d17 None admin] RBAC: Authorizing `identity:get_project(project_id=45517a954fcb4bfe9bc89c2cf3ff487f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:19.652554 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f054c33f-e5cd-4f65-8927-1ddd81646d17 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:19.652554 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 559/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 04:00:19 2022] GET /identity/v3/projects/45517a954fcb4bfe9bc89c2cf3ff487f => generated 311 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:19.660739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f678b1-8c53-43ac-950b-28f67b800df4 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:19.751935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:19.768939 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f678b1-8c53-43ac-950b-28f67b800df4 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:19.808987 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999118b-2ace-46b2-88a9-f78e1e332525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:19.808987 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999118b-2ace-46b2-88a9-f78e1e332525 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:19.808987 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999118b-2ace-46b2-88a9-f78e1e332525 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:19.809401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999118b-2ace-46b2-88a9-f78e1e332525 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:19.810814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999118b-2ace-46b2-88a9-f78e1e332525 None admin] RBAC: Adding query filter params (role.id=a048041c27f44edf98a110d23138a327, scope.project.id=45517a954fcb4bfe9bc89c2cf3ff487f, user.id=37ee96a2cbc649aea5fff5cd56ed35f6) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:19.810814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999118b-2ace-46b2-88a9-f78e1e332525 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:19.812364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999118b-2ace-46b2-88a9-f78e1e332525 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:19.818160 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 571/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 04:00:19 2022] GET /identity/v3/role_assignments?user.id=37ee96a2cbc649aea5fff5cd56ed35f6&scope.project.id=45517a954fcb4bfe9bc89c2cf3ff487f&role.id=a048041c27f44edf98a110d23138a327 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:21.912742 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370ad59-c5d7-464a-ad42-f6ae4451e388 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:21.913311 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370ad59-c5d7-464a-ad42-f6ae4451e388 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:21.913311 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370ad59-c5d7-464a-ad42-f6ae4451e388 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:21.914709 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 560/1131] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:00:21.919878 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df1d71-ad47-4a45-95c0-0598fb898d3a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:21.920255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df1d71-ad47-4a45-95c0-0598fb898d3a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:21.920443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df1d71-ad47-4a45-95c0-0598fb898d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:21.936818 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b7df1d71-ad47-4a45-95c0-0598fb898d3a None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:21.937980 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b7df1d71-ad47-4a45-95c0-0598fb898d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:22.002801 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 572/1132] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:00:22.026602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370ad59-c5d7-464a-ad42-f6ae4451e388 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:22.126866 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a370ad59-c5d7-464a-ad42-f6ae4451e388 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:22.160811 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f786a-0afe-4d9b-9d10-0235e3f8b630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:22.161586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f786a-0afe-4d9b-9d10-0235e3f8b630 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:22.161869 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f786a-0afe-4d9b-9d10-0235e3f8b630 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:22.162146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f786a-0afe-4d9b-9d10-0235e3f8b630 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:22.163402 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f786a-0afe-4d9b-9d10-0235e3f8b630 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:22.164777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f786a-0afe-4d9b-9d10-0235e3f8b630 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:22.166390 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 561/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:00:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:22.172202 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df1d71-ad47-4a45-95c0-0598fb898d3a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:22.247117 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:22.268349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df1d71-ad47-4a45-95c0-0598fb898d3a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:22.302472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe2b47-8f32-419c-9ba6-eff066c2eb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:22.303126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe2b47-8f32-419c-9ba6-eff066c2eb5c None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:22.303414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe2b47-8f32-419c-9ba6-eff066c2eb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:22.303695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe2b47-8f32-419c-9ba6-eff066c2eb5c None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:22.304409 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe2b47-8f32-419c-9ba6-eff066c2eb5c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:22.305829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe2b47-8f32-419c-9ba6-eff066c2eb5c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:22.424283 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 573/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jun 24 04:00:22 2022] POST /identity/v3/projects => generated 311 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:00:24.489289 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56f04b-ee24-4bc2-91f7-8d69f5c976d3 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:24.489823 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56f04b-ee24-4bc2-91f7-8d69f5c976d3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:24.489823 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf56f04b-ee24-4bc2-91f7-8d69f5c976d3 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:24.491236 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 562/1135] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:00:24.496585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f274ad2-284e-458e-9512-4656226d4a2c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:24.497041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f274ad2-284e-458e-9512-4656226d4a2c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:24.497136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f274ad2-284e-458e-9512-4656226d4a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:24.514141 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5f274ad2-284e-458e-9512-4656226d4a2c None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:24.515830 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5f274ad2-284e-458e-9512-4656226d4a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:24.580021 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 574/1136] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:00:24.605508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf56f04b-ee24-4bc2-91f7-8d69f5c976d3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:24.667269 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:24.713964 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf56f04b-ee24-4bc2-91f7-8d69f5c976d3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:24.747615 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb52648-029a-4b6c-ba0f-1b2ae6063205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:24.748328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb52648-029a-4b6c-ba0f-1b2ae6063205 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:24.748613 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb52648-029a-4b6c-ba0f-1b2ae6063205 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:24.748889 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb52648-029a-4b6c-ba0f-1b2ae6063205 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:24.750228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb52648-029a-4b6c-ba0f-1b2ae6063205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:24.751685 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb52648-029a-4b6c-ba0f-1b2ae6063205 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:24.753304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 563/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:00:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:24.759107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f274ad2-284e-458e-9512-4656226d4a2c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:24.816684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:24.855452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f274ad2-284e-458e-9512-4656226d4a2c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:24.889308 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bacae0-4cfb-43cf-bac7-67a29c73a217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:24.890905 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bacae0-4cfb-43cf-bac7-67a29c73a217 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:24.891154 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bacae0-4cfb-43cf-bac7-67a29c73a217 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:24.891456 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bacae0-4cfb-43cf-bac7-67a29c73a217 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:24.892176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bacae0-4cfb-43cf-bac7-67a29c73a217 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:24.893612 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bacae0-4cfb-43cf-bac7-67a29c73a217 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:25.003034 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 575/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jun 24 04:00:24 2022] POST /identity/v3/users => generated 286 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:00:27.154129 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ceafd-8053-4bd0-ba6c-7dd5dffcdb4c None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:27.154687 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ceafd-8053-4bd0-ba6c-7dd5dffcdb4c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:27.154774 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ceafd-8053-4bd0-ba6c-7dd5dffcdb4c None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:27.156738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 564/1139] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:00:27.162064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d060aa-4d42-4085-aa01-7fb1b27c97f7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:27.162521 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d060aa-4d42-4085-aa01-7fb1b27c97f7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:27.162660 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d060aa-4d42-4085-aa01-7fb1b27c97f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:27.178858 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-99d060aa-4d42-4085-aa01-7fb1b27c97f7 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:27.180041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-99d060aa-4d42-4085-aa01-7fb1b27c97f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:27.245806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 576/1140] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:00:27.269661 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ceafd-8053-4bd0-ba6c-7dd5dffcdb4c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:27.368537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ceafd-8053-4bd0-ba6c-7dd5dffcdb4c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:27.394481 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:27.401755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bee103-4587-4d08-97a8-57e3ca535559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:27.404874 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bee103-4587-4d08-97a8-57e3ca535559 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:27.404874 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bee103-4587-4d08-97a8-57e3ca535559 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:27.404874 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bee103-4587-4d08-97a8-57e3ca535559 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:27.413695 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bee103-4587-4d08-97a8-57e3ca535559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 04:00:27.422059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-47bee103-4587-4d08-97a8-57e3ca535559 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:00:27.422059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:00:27.422059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:00:27.422059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:00:27.422059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:00:27.422059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:00:27.422059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:00:27.422059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:00:27.422059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:00:27.422059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:00:27.422059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:00:27.422059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:00:27.422059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 04:00:27.422059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 04:00:27.422059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 04:00:27.422059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 04:00:27.423221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 04:00:27.423221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 04:00:27.423221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 04:00:27.423221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 04:00:27.423221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:00:27.423221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:00:27.423221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:00:27.423221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:00:27.423221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:00:27.423221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:00:27.423221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:00:27.423221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:00:27.423221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:00:27.423221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:00:27.423221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:00:27.423221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:00:27.424165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:00:27.424165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:00:27.424165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:00:27.424165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:00:27.424165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 04:00:27.424165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 04:00:27.424165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 04:00:27.424165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 04:00:27.424165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 04:00:27.424165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 04:00:27.424165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:00:27.424165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:00:27.424165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 565/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 04:00:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:00:27.429705 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d060aa-4d42-4085-aa01-7fb1b27c97f7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:27.475767 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:27.527651 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d060aa-4d42-4085-aa01-7fb1b27c97f7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:27.565465 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bce548-236b-4e14-8399-138fd21ec310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:27.566134 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bce548-236b-4e14-8399-138fd21ec310 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:27.566447 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bce548-236b-4e14-8399-138fd21ec310 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:27.566727 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bce548-236b-4e14-8399-138fd21ec310 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:27.567473 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bce548-236b-4e14-8399-138fd21ec310 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:27.567785 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bce548-236b-4e14-8399-138fd21ec310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:27.569160 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bce548-236b-4e14-8399-138fd21ec310 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:27.578097 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 577/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 04:00:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:27.584034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bee103-4587-4d08-97a8-57e3ca535559 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:27.682218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bee103-4587-4d08-97a8-57e3ca535559 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:27.715475 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3001495-405d-482f-8400-a32f1786c9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:27.716333 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3001495-405d-482f-8400-a32f1786c9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:27.716616 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3001495-405d-482f-8400-a32f1786c9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:27.716892 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3001495-405d-482f-8400-a32f1786c9f2 None admin] PATH_INFO: `/v3/users/fa5f935d8f0943ccad20f06adfe969c5` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:27.730120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3001495-405d-482f-8400-a32f1786c9f2 None admin] RBAC: Authorizing `identity:get_user(user_id=fa5f935d8f0943ccad20f06adfe969c5)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:27.731617 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3001495-405d-482f-8400-a32f1786c9f2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:27.733302 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 566/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 04:00:27 2022] GET /identity/v3/users/fa5f935d8f0943ccad20f06adfe969c5 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:27.740590 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bce548-236b-4e14-8399-138fd21ec310 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:27.840022 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bce548-236b-4e14-8399-138fd21ec310 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:27.849986 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:27.873616 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69dd20-3dc6-4bf2-9e5d-b983d336dcf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:27.875272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69dd20-3dc6-4bf2-9e5d-b983d336dcf1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:27.875562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69dd20-3dc6-4bf2-9e5d-b983d336dcf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:27.875840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69dd20-3dc6-4bf2-9e5d-b983d336dcf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:27.941955 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69dd20-3dc6-4bf2-9e5d-b983d336dcf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:27.943363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69dd20-3dc6-4bf2-9e5d-b983d336dcf1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:28.038494 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 578/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 04:00:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 04:00:28.044284 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3001495-405d-482f-8400-a32f1786c9f2 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:28.090196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:28.142340 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3001495-405d-482f-8400-a32f1786c9f2 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:28.175844 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eeff30-20b8-4341-9d2e-83b36e688cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:28.176691 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eeff30-20b8-4341-9d2e-83b36e688cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:28.176977 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eeff30-20b8-4341-9d2e-83b36e688cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:28.177255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eeff30-20b8-4341-9d2e-83b36e688cc1 None admin] PATH_INFO: `/v3/projects/1d207bed7021431f91c9056a899a5e49` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:28.178460 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eeff30-20b8-4341-9d2e-83b36e688cc1 None admin] RBAC: Authorizing `identity:get_project(project_id=1d207bed7021431f91c9056a899a5e49)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:28.179901 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eeff30-20b8-4341-9d2e-83b36e688cc1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:28.181600 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 567/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 04:00:28 2022] GET /identity/v3/projects/1d207bed7021431f91c9056a899a5e49 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:28.188204 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69dd20-3dc6-4bf2-9e5d-b983d336dcf1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:28.198073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:28.285176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69dd20-3dc6-4bf2-9e5d-b983d336dcf1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:28.318614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83efd9a9-f57f-48ae-9e60-fd818ef53dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:28.319272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efd9a9-f57f-48ae-9e60-fd818ef53dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:28.320737 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efd9a9-f57f-48ae-9e60-fd818ef53dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:28.320737 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83efd9a9-f57f-48ae-9e60-fd818ef53dc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:28.321178 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efd9a9-f57f-48ae-9e60-fd818ef53dc5 None admin] RBAC: Adding query filter params (role.id=f0f7e28cbe314921afdfb657b163436f, scope.project.id=1d207bed7021431f91c9056a899a5e49, user.id=fa5f935d8f0943ccad20f06adfe969c5) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:28.321497 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efd9a9-f57f-48ae-9e60-fd818ef53dc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:28.323418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83efd9a9-f57f-48ae-9e60-fd818ef53dc5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:28.329436 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 579/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 04:00:28 2022] GET /identity/v3/role_assignments?user.id=fa5f935d8f0943ccad20f06adfe969c5&scope.project.id=1d207bed7021431f91c9056a899a5e49&role.id=f0f7e28cbe314921afdfb657b163436f => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:00:30.350722 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8621635-a57e-4c17-ad06-6d6ddb920c06 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:30.351257 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8621635-a57e-4c17-ad06-6d6ddb920c06 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:30.351257 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8621635-a57e-4c17-ad06-6d6ddb920c06 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:30.352567 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 568/1147] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:00:30.357832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb0b45-0d69-4342-8dcc-4aee7fe5a9f9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:30.358252 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb0b45-0d69-4342-8dcc-4aee7fe5a9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:30.358458 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb0b45-0d69-4342-8dcc-4aee7fe5a9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:30.374941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-78cb0b45-0d69-4342-8dcc-4aee7fe5a9f9 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:30.376112 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-78cb0b45-0d69-4342-8dcc-4aee7fe5a9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:30.439549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 580/1148] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:00:30.462184 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8621635-a57e-4c17-ad06-6d6ddb920c06 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:30.559791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8621635-a57e-4c17-ad06-6d6ddb920c06 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:30.573684 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:30.592972 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb0b0c-68aa-4b9f-8aec-21b6e92392b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:30.605168 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb0b0c-68aa-4b9f-8aec-21b6e92392b8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:30.605168 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb0b0c-68aa-4b9f-8aec-21b6e92392b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:30.605168 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb0b0c-68aa-4b9f-8aec-21b6e92392b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:30.605168 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bb0b0c-68aa-4b9f-8aec-21b6e92392b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 04:00:30.611397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-92bb0b0c-68aa-4b9f-8aec-21b6e92392b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:00:30.611397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:00:30.611397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:00:30.611397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:00:30.611397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:00:30.611397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:00:30.611397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:00:30.611397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:00:30.611397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:00:30.611397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:00:30.611397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:00:30.611397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:00:30.611397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 04:00:30.611397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 04:00:30.611397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 04:00:30.611397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 04:00:30.612647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 04:00:30.612647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 04:00:30.612647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 04:00:30.612647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 04:00:30.612647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:00:30.612647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:00:30.612647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:00:30.612647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:00:30.612647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:00:30.612647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:00:30.612647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:00:30.612647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:00:30.612647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:00:30.612647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:00:30.612647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:00:30.612647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:00:30.613650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:00:30.613650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:00:30.613650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:00:30.613650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:00:30.613650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 04:00:30.613650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 04:00:30.613650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 04:00:30.613650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 04:00:30.613650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 04:00:30.613650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 04:00:30.613650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:00:30.613650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:00:30.613650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 569/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 04:00:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:00:30.618735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb0b45-0d69-4342-8dcc-4aee7fe5a9f9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:30.621128 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:30.714762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb0b45-0d69-4342-8dcc-4aee7fe5a9f9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:30.746618 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41233a-4199-45a0-8f0f-72991b0ff2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:30.747274 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41233a-4199-45a0-8f0f-72991b0ff2be None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:30.747557 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41233a-4199-45a0-8f0f-72991b0ff2be None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:30.747833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41233a-4199-45a0-8f0f-72991b0ff2be None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:30.748568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41233a-4199-45a0-8f0f-72991b0ff2be None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:30.748876 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41233a-4199-45a0-8f0f-72991b0ff2be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:30.751090 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41233a-4199-45a0-8f0f-72991b0ff2be None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:30.758819 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 581/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 04:00:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:30.764475 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb0b0c-68aa-4b9f-8aec-21b6e92392b8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:30.862425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb0b0c-68aa-4b9f-8aec-21b6e92392b8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:30.895843 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea9b7a-84fe-4dc0-8258-d629aba66839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:30.896689 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea9b7a-84fe-4dc0-8258-d629aba66839 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:30.896975 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea9b7a-84fe-4dc0-8258-d629aba66839 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:30.897256 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea9b7a-84fe-4dc0-8258-d629aba66839 None admin] PATH_INFO: `/v3/users/fa5f935d8f0943ccad20f06adfe969c5` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:30.898464 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea9b7a-84fe-4dc0-8258-d629aba66839 None admin] RBAC: Authorizing `identity:get_user(user_id=fa5f935d8f0943ccad20f06adfe969c5)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:30.899905 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea9b7a-84fe-4dc0-8258-d629aba66839 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:30.901510 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 570/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 04:00:30 2022] GET /identity/v3/users/fa5f935d8f0943ccad20f06adfe969c5 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:30.907283 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41233a-4199-45a0-8f0f-72991b0ff2be None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:30.967216 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:31.004765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41233a-4199-45a0-8f0f-72991b0ff2be None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:31.036455 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526e6c92-1035-4539-98db-5c3044a55413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:31.037313 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e6c92-1035-4539-98db-5c3044a55413 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:31.037600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e6c92-1035-4539-98db-5c3044a55413 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:31.037880 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526e6c92-1035-4539-98db-5c3044a55413 None admin] PATH_INFO: `/v3/projects/1d207bed7021431f91c9056a899a5e49` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:31.039149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e6c92-1035-4539-98db-5c3044a55413 None admin] RBAC: Authorizing `identity:get_project(project_id=1d207bed7021431f91c9056a899a5e49)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:31.040588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526e6c92-1035-4539-98db-5c3044a55413 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:31.042425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 582/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 04:00:30 2022] GET /identity/v3/projects/1d207bed7021431f91c9056a899a5e49 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:31.048991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea9b7a-84fe-4dc0-8258-d629aba66839 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:31.092039 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:31.149680 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea9b7a-84fe-4dc0-8258-d629aba66839 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:31.182532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2fada-4735-471f-aac3-c73307d8bb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:31.183401 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2fada-4735-471f-aac3-c73307d8bb16 None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:31.183680 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2fada-4735-471f-aac3-c73307d8bb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:31.183953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2fada-4735-471f-aac3-c73307d8bb16 None admin] PATH_INFO: `/v3/projects/1d207bed7021431f91c9056a899a5e49/users/fa5f935d8f0943ccad20f06adfe969c5/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:31.185873 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2fada-4735-471f-aac3-c73307d8bb16 None admin] RBAC: Authorizing `identity:create_grant(project_id=1d207bed7021431f91c9056a899a5e49, user_id=fa5f935d8f0943ccad20f06adfe969c5, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:31.187884 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b2fada-4735-471f-aac3-c73307d8bb16 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:31.266042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 571/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 04:00:31 2022] PUT /identity/v3/projects/1d207bed7021431f91c9056a899a5e49/users/fa5f935d8f0943ccad20f06adfe969c5/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:00:33.341673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8552bb-c959-406c-95a9-33ab5cb8f3de None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:33.342191 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8552bb-c959-406c-95a9-33ab5cb8f3de None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:33.342191 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8552bb-c959-406c-95a9-33ab5cb8f3de None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:33.344337 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 583/1154] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:00:33.349586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9590538-0b6b-4ce6-9d0d-692ff1b344ea None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:33.349985 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9590538-0b6b-4ce6-9d0d-692ff1b344ea None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:33.350144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9590538-0b6b-4ce6-9d0d-692ff1b344ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:33.366820 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f9590538-0b6b-4ce6-9d0d-692ff1b344ea None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:33.367989 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f9590538-0b6b-4ce6-9d0d-692ff1b344ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:33.434582 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 572/1155] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:00:33.464442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8552bb-c959-406c-95a9-33ab5cb8f3de None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:33.563263 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8552bb-c959-406c-95a9-33ab5cb8f3de None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:33.593364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:33.598147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4183e4-c4e5-49a8-80e3-85eab4e36b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:33.598964 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4183e4-c4e5-49a8-80e3-85eab4e36b24 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:33.602161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4183e4-c4e5-49a8-80e3-85eab4e36b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:33.602161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4183e4-c4e5-49a8-80e3-85eab4e36b24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:33.609558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4183e4-c4e5-49a8-80e3-85eab4e36b24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 04:00:33.620564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-ec4183e4-c4e5-49a8-80e3-85eab4e36b24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:00:33.620564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:00:33.620564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:00:33.620564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:00:33.620564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:00:33.620564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:00:33.620564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:00:33.620564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:00:33.620564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:00:33.620564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:00:33.620564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:00:33.620564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:00:33.620564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 04:00:33.620564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 04:00:33.620564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 04:00:33.620564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 04:00:33.621736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 04:00:33.621736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 04:00:33.621736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 04:00:33.621736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 04:00:33.621736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:00:33.621736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:00:33.621736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:00:33.621736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:00:33.621736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:00:33.621736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:00:33.621736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:00:33.621736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:00:33.621736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:00:33.621736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:00:33.621736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:00:33.621736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:00:33.622750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:00:33.622750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:00:33.622750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:00:33.622750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:00:33.622750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 04:00:33.622750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 04:00:33.622750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 04:00:33.622750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 04:00:33.622750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 04:00:33.622750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 04:00:33.622750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:00:33.622750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:00:33.622750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 584/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 04:00:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:00:33.629302 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9590538-0b6b-4ce6-9d0d-692ff1b344ea None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:33.660058 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:33.728350 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9590538-0b6b-4ce6-9d0d-692ff1b344ea None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:33.761616 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebd522-356d-4f6a-9c96-4cf60a0de89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:33.762275 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebd522-356d-4f6a-9c96-4cf60a0de89b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:33.762587 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebd522-356d-4f6a-9c96-4cf60a0de89b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:33.762864 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebd522-356d-4f6a-9c96-4cf60a0de89b None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:33.763601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ebd522-356d-4f6a-9c96-4cf60a0de89b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:33.763907 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ebd522-356d-4f6a-9c96-4cf60a0de89b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:33.765926 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ebd522-356d-4f6a-9c96-4cf60a0de89b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:33.773806 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 573/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 04:00:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:33.779598 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4183e4-c4e5-49a8-80e3-85eab4e36b24 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:33.875788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4183e4-c4e5-49a8-80e3-85eab4e36b24 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:33.908149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb456fb-32f9-4f52-a9d5-4d0d0fb11424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:33.908993 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb456fb-32f9-4f52-a9d5-4d0d0fb11424 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:33.909275 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb456fb-32f9-4f52-a9d5-4d0d0fb11424 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:33.909544 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb456fb-32f9-4f52-a9d5-4d0d0fb11424 None admin] PATH_INFO: `/v3/users/fa5f935d8f0943ccad20f06adfe969c5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:33.910807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb456fb-32f9-4f52-a9d5-4d0d0fb11424 None admin] RBAC: Authorizing `identity:get_user(user_id=fa5f935d8f0943ccad20f06adfe969c5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:33.912217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb456fb-32f9-4f52-a9d5-4d0d0fb11424 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:33.913796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 585/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 04:00:33 2022] GET /identity/v3/users/fa5f935d8f0943ccad20f06adfe969c5 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:33.919537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebd522-356d-4f6a-9c96-4cf60a0de89b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:34.011400 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:34.017923 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebd522-356d-4f6a-9c96-4cf60a0de89b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:34.050602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ad614-aa00-47b4-a11a-57cd7d4a44fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:34.051249 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ad614-aa00-47b4-a11a-57cd7d4a44fd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:34.051534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ad614-aa00-47b4-a11a-57cd7d4a44fd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:34.051812 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ad614-aa00-47b4-a11a-57cd7d4a44fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:34.118526 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ad614-aa00-47b4-a11a-57cd7d4a44fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:34.120243 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ad614-aa00-47b4-a11a-57cd7d4a44fd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:34.209681 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:34.217553 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 574/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 04:00:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 04:00:34.223976 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb456fb-32f9-4f52-a9d5-4d0d0fb11424 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:34.282455 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:34.322295 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb456fb-32f9-4f52-a9d5-4d0d0fb11424 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:34.355474 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f3d35-c5a2-44a6-808d-4e052bb41539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:34.356335 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f3d35-c5a2-44a6-808d-4e052bb41539 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:34.356620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f3d35-c5a2-44a6-808d-4e052bb41539 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:34.356894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f3d35-c5a2-44a6-808d-4e052bb41539 None admin] PATH_INFO: `/v3/projects/1d207bed7021431f91c9056a899a5e49` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:34.358067 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f3d35-c5a2-44a6-808d-4e052bb41539 None admin] RBAC: Authorizing `identity:get_project(project_id=1d207bed7021431f91c9056a899a5e49)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:34.360166 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f3d35-c5a2-44a6-808d-4e052bb41539 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:34.361851 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 586/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 04:00:34 2022] GET /identity/v3/projects/1d207bed7021431f91c9056a899a5e49 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:34.367950 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ad614-aa00-47b4-a11a-57cd7d4a44fd None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:34.469134 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ad614-aa00-47b4-a11a-57cd7d4a44fd None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:34.504727 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add308e-3bb0-4c98-b729-d5ee7a794dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:34.505381 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add308e-3bb0-4c98-b729-d5ee7a794dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:34.505667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add308e-3bb0-4c98-b729-d5ee7a794dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:34.505946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add308e-3bb0-4c98-b729-d5ee7a794dd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:34.506796 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add308e-3bb0-4c98-b729-d5ee7a794dd5 None admin] RBAC: Adding query filter params (role.id=f0f7e28cbe314921afdfb657b163436f, scope.project.id=1d207bed7021431f91c9056a899a5e49, user.id=fa5f935d8f0943ccad20f06adfe969c5) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:34.507114 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add308e-3bb0-4c98-b729-d5ee7a794dd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:34.508546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add308e-3bb0-4c98-b729-d5ee7a794dd5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:34.514275 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 575/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 04:00:34 2022] GET /identity/v3/role_assignments?user.id=fa5f935d8f0943ccad20f06adfe969c5&scope.project.id=1d207bed7021431f91c9056a899a5e49&role.id=f0f7e28cbe314921afdfb657b163436f => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:36.557360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c744ee-8e58-4b82-b204-817c44b13a90 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:36.557929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c744ee-8e58-4b82-b204-817c44b13a90 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:36.557929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c744ee-8e58-4b82-b204-817c44b13a90 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:36.559329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 587/1162] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:00:36.564550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d90949-3e6e-4ea2-a6c1-5929576dc963 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:36.564970 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d90949-3e6e-4ea2-a6c1-5929576dc963 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:36.565111 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d90949-3e6e-4ea2-a6c1-5929576dc963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:36.581467 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-44d90949-3e6e-4ea2-a6c1-5929576dc963 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:36.582686 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-44d90949-3e6e-4ea2-a6c1-5929576dc963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:36.635326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:36.648522 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 576/1163] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:00:36.672238 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c744ee-8e58-4b82-b204-817c44b13a90 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:36.771365 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c744ee-8e58-4b82-b204-817c44b13a90 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:36.797783 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:36.805215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90ec2e4-6285-4aa5-ba86-cdfcadd973f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:36.805859 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ec2e4-6285-4aa5-ba86-cdfcadd973f2 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:36.806144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ec2e4-6285-4aa5-ba86-cdfcadd973f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:36.811070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ec2e4-6285-4aa5-ba86-cdfcadd973f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:36.871462 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90ec2e4-6285-4aa5-ba86-cdfcadd973f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:36.872906 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90ec2e4-6285-4aa5-ba86-cdfcadd973f2 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:36.967337 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 588/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 04:00:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 04:00:36.973895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d90949-3e6e-4ea2-a6c1-5929576dc963 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:37.070419 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d90949-3e6e-4ea2-a6c1-5929576dc963 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:37.103773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2004b28-10d4-4df1-bc5f-a61280a897a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:37.104562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2004b28-10d4-4df1-bc5f-a61280a897a0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:37.104846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2004b28-10d4-4df1-bc5f-a61280a897a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:37.105122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2004b28-10d4-4df1-bc5f-a61280a897a0 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:37.111933 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2004b28-10d4-4df1-bc5f-a61280a897a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:37.113371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2004b28-10d4-4df1-bc5f-a61280a897a0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:37.127764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-b2004b28-10d4-4df1-bc5f-a61280a897a0 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 04:00:37.127764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:00:37.127764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 24 04:00:37.127764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 24 04:00:37.127764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 04:00:37.127764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 04:00:37.127764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 04:00:37.127764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 04:00:37.127764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 24 04:00:37.127764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:00:37.127764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 04:00:37.127764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:00:37.127764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:00:37.127764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:00:37.127764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:00:37.127764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:00:37.127764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:00:37.129017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:00:37.129017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:00:37.129017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:00:37.129017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:00:37.129017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:00:37.129017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:00:37.129017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 04:00:37.129017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 04:00:37.129017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 04:00:37.129017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 04:00:37.129017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:00:37.129017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:00:37.129017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:00:37.129017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:00:37.129017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:00:37.129017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:00:37.129908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:00:37.129908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:00:37.129908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:00:37.129908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:00:37.129908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:00:37.129908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:00:37.129908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:00:37.129908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:00:37.129908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:00:37.129908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:00:37.129908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 24 04:00:37.129908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 04:00:37.129908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 04:00:37.129908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:00:37.129908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 577/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jun 24 04:00:36 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:00:37.134848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90ec2e4-6285-4aa5-ba86-cdfcadd973f2 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:37.164227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:37.234354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90ec2e4-6285-4aa5-ba86-cdfcadd973f2 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:37.267747 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935c09b7-0eeb-4e16-a87f-eefc463d8f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:37.268392 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c09b7-0eeb-4e16-a87f-eefc463d8f9a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:37.268673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c09b7-0eeb-4e16-a87f-eefc463d8f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:37.268948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935c09b7-0eeb-4e16-a87f-eefc463d8f9a None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:37.269661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c09b7-0eeb-4e16-a87f-eefc463d8f9a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:37.269965 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c09b7-0eeb-4e16-a87f-eefc463d8f9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:37.271356 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935c09b7-0eeb-4e16-a87f-eefc463d8f9a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:37.278226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 589/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Jun 24 04:00:37 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:39.283121 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf97d9-5cae-488f-8d37-3c3dbafa73ad None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:39.283676 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf97d9-5cae-488f-8d37-3c3dbafa73ad None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:39.283676 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf97d9-5cae-488f-8d37-3c3dbafa73ad None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:39.285044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 578/1167] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:00:39.290132 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1058cb7-7994-4294-8ef8-34c656691020 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:39.290545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1058cb7-7994-4294-8ef8-34c656691020 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:39.290724 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1058cb7-7994-4294-8ef8-34c656691020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:39.307121 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a1058cb7-7994-4294-8ef8-34c656691020 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:39.308244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a1058cb7-7994-4294-8ef8-34c656691020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:39.373156 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 590/1168] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:00:39.396236 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf97d9-5cae-488f-8d37-3c3dbafa73ad None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:39.445841 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:39.496513 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf97d9-5cae-488f-8d37-3c3dbafa73ad None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:39.529122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e83d2-1a91-48e9-8326-a37e64b9cb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:39.529765 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e83d2-1a91-48e9-8326-a37e64b9cb2b None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:39.530890 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e83d2-1a91-48e9-8326-a37e64b9cb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:39.531031 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e83d2-1a91-48e9-8326-a37e64b9cb2b None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:39.531760 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e83d2-1a91-48e9-8326-a37e64b9cb2b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:39.533122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e83d2-1a91-48e9-8326-a37e64b9cb2b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:39.627575 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 579/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Fri Jun 24 04:00:39 2022] POST /identity/v3/domains => generated 267 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:00:41.713120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14153ab-12df-4de8-9259-bd6b0832fa02 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:41.713663 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14153ab-12df-4de8-9259-bd6b0832fa02 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:41.713783 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14153ab-12df-4de8-9259-bd6b0832fa02 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:41.715202 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 591/1170] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:00:41.721170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4766c-b589-45df-9191-8bd77e5c7911 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:41.721609 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4766c-b589-45df-9191-8bd77e5c7911 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:41.721737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4766c-b589-45df-9191-8bd77e5c7911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:41.738074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-57f4766c-b589-45df-9191-8bd77e5c7911 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:41.739288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-57f4766c-b589-45df-9191-8bd77e5c7911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:41.804457 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 580/1171] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:00:41.829343 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14153ab-12df-4de8-9259-bd6b0832fa02 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:41.852974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:41.928221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14153ab-12df-4de8-9259-bd6b0832fa02 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:41.961885 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0977196-1a9d-4055-ba2e-f5d589982681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:41.962765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0977196-1a9d-4055-ba2e-f5d589982681 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:41.963060 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0977196-1a9d-4055-ba2e-f5d589982681 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:41.963390 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0977196-1a9d-4055-ba2e-f5d589982681 None admin] PATH_INFO: `/v3/domains/661cdb96cde743798bb6a9b0db2e4901` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:41.973510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0977196-1a9d-4055-ba2e-f5d589982681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=661cdb96cde743798bb6a9b0db2e4901)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:41.974996 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0977196-1a9d-4055-ba2e-f5d589982681 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:41.976619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 592/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Fri Jun 24 04:00:41 2022] GET /identity/v3/domains/661cdb96cde743798bb6a9b0db2e4901 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:41.982429 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4766c-b589-45df-9191-8bd77e5c7911 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:42.021542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:42.080300 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4766c-b589-45df-9191-8bd77e5c7911 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:42.113065 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc60b5-22dd-46ac-b7ad-879c824dd2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:42.113716 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc60b5-22dd-46ac-b7ad-879c824dd2c5 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:42.114002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc60b5-22dd-46ac-b7ad-879c824dd2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:42.114282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc60b5-22dd-46ac-b7ad-879c824dd2c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:42.115023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc60b5-22dd-46ac-b7ad-879c824dd2c5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:42.116452 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc60b5-22dd-46ac-b7ad-879c824dd2c5 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:42.192552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 581/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jun 24 04:00:41 2022] POST /identity/v3/projects => generated 361 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:00:44.253049 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f8ee5-f49d-469c-b431-8edd4383eaf9 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:44.253595 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f8ee5-f49d-469c-b431-8edd4383eaf9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:44.253595 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f8ee5-f49d-469c-b431-8edd4383eaf9 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:44.255010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 593/1174] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:00:44.260323 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7de127-4502-4abb-9007-727dbe33cc81 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:44.260730 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7de127-4502-4abb-9007-727dbe33cc81 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:44.260876 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7de127-4502-4abb-9007-727dbe33cc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:44.278230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9f7de127-4502-4abb-9007-727dbe33cc81 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:44.279440 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9f7de127-4502-4abb-9007-727dbe33cc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:44.344058 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 582/1175] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:00:44.368586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f8ee5-f49d-469c-b431-8edd4383eaf9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:44.451499 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:44.469852 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f8ee5-f49d-469c-b431-8edd4383eaf9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:44.504205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90aef10-a950-4e93-8c70-621ec0e1f46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:44.505068 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90aef10-a950-4e93-8c70-621ec0e1f46c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:44.505357 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90aef10-a950-4e93-8c70-621ec0e1f46c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:44.505639 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90aef10-a950-4e93-8c70-621ec0e1f46c None admin] PATH_INFO: `/v3/domains/661cdb96cde743798bb6a9b0db2e4901` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:44.506845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90aef10-a950-4e93-8c70-621ec0e1f46c None admin] RBAC: Authorizing `identity:get_domain(domain_id=661cdb96cde743798bb6a9b0db2e4901)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:44.508277 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90aef10-a950-4e93-8c70-621ec0e1f46c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:44.509881 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 594/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Fri Jun 24 04:00:44 2022] GET /identity/v3/domains/661cdb96cde743798bb6a9b0db2e4901 => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:44.516122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7de127-4502-4abb-9007-727dbe33cc81 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:44.536879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:44.613991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7de127-4502-4abb-9007-727dbe33cc81 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:44.647496 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951e6e6-57dc-4299-a53d-74a29df194c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:44.648154 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951e6e6-57dc-4299-a53d-74a29df194c2 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:44.648435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951e6e6-57dc-4299-a53d-74a29df194c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:44.648708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951e6e6-57dc-4299-a53d-74a29df194c2 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:44.649401 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951e6e6-57dc-4299-a53d-74a29df194c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:44.650849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951e6e6-57dc-4299-a53d-74a29df194c2 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:44.753825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 583/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jun 24 04:00:44 2022] POST /identity/v3/users => generated 311 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:00:46.806348 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e90ca-45b8-4bc9-930c-2187d9a59707 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:46.806906 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e90ca-45b8-4bc9-930c-2187d9a59707 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:46.806995 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e90ca-45b8-4bc9-930c-2187d9a59707 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:46.808295 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 595/1178] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:00:46.813667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838e71b-bc96-49d3-88c8-775a27c0a2b3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:46.814089 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838e71b-bc96-49d3-88c8-775a27c0a2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:46.814231 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838e71b-bc96-49d3-88c8-775a27c0a2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:46.831753 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d838e71b-bc96-49d3-88c8-775a27c0a2b3 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:46.832932 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d838e71b-bc96-49d3-88c8-775a27c0a2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:46.913590 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 584/1179] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:00:46.936553 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e90ca-45b8-4bc9-930c-2187d9a59707 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:47.034612 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e90ca-45b8-4bc9-930c-2187d9a59707 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:47.068835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14826b7a-da03-4491-95a2-66bb3ed8ab46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:47.069557 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14826b7a-da03-4491-95a2-66bb3ed8ab46 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:47.069840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14826b7a-da03-4491-95a2-66bb3ed8ab46 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:47.070116 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14826b7a-da03-4491-95a2-66bb3ed8ab46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:47.080973 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14826b7a-da03-4491-95a2-66bb3ed8ab46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 04:00:47.088500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-14826b7a-da03-4491-95a2-66bb3ed8ab46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:00:47.088500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:00:47.088500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:00:47.088500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:00:47.088500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:00:47.088500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:00:47.088500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:00:47.088500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:00:47.088500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:00:47.088500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:00:47.088500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:00:47.088500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:00:47.088500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 04:00:47.088500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 04:00:47.088500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 04:00:47.088500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 04:00:47.089558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 04:00:47.089558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 04:00:47.089558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 04:00:47.089558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 04:00:47.089558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:00:47.089558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:00:47.089558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:00:47.089558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:00:47.089558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:00:47.089558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:00:47.089558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:00:47.089558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:00:47.089558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:00:47.089558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:00:47.089558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:00:47.089558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:00:47.090642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:00:47.090642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:00:47.090642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:00:47.090642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:00:47.090642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 04:00:47.090642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 04:00:47.090642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 04:00:47.090642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 04:00:47.090642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 04:00:47.090642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 04:00:47.090642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:00:47.090642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:00:47.090642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 596/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 04:00:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:00:47.095801 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838e71b-bc96-49d3-88c8-775a27c0a2b3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:47.103735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:47.194038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838e71b-bc96-49d3-88c8-775a27c0a2b3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:47.226885 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ef491-8a84-4b5c-b0e6-bdc1e2b9a33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:47.227540 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ef491-8a84-4b5c-b0e6-bdc1e2b9a33c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:47.227824 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ef491-8a84-4b5c-b0e6-bdc1e2b9a33c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:47.228102 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ef491-8a84-4b5c-b0e6-bdc1e2b9a33c None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:47.228843 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ef491-8a84-4b5c-b0e6-bdc1e2b9a33c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:47.229151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ef491-8a84-4b5c-b0e6-bdc1e2b9a33c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:47.230549 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ef491-8a84-4b5c-b0e6-bdc1e2b9a33c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:47.238697 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 585/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 04:00:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:47.244598 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14826b7a-da03-4491-95a2-66bb3ed8ab46 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:47.258305 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:47.343398 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14826b7a-da03-4491-95a2-66bb3ed8ab46 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:47.376122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f69e7-3bfc-4b63-b3a5-0de755683996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:47.376984 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f69e7-3bfc-4b63-b3a5-0de755683996 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:47.377271 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f69e7-3bfc-4b63-b3a5-0de755683996 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:47.377656 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f69e7-3bfc-4b63-b3a5-0de755683996 None admin] PATH_INFO: `/v3/users/d0027c47d8ec4342ba162f994420f8cf` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:47.390929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f69e7-3bfc-4b63-b3a5-0de755683996 None admin] RBAC: Authorizing `identity:get_user(user_id=d0027c47d8ec4342ba162f994420f8cf)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:47.392390 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f69e7-3bfc-4b63-b3a5-0de755683996 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:47.394008 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 597/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 04:00:47 2022] GET /identity/v3/users/d0027c47d8ec4342ba162f994420f8cf => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:47.399686 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ef491-8a84-4b5c-b0e6-bdc1e2b9a33c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:47.475690 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:47.510340 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ef491-8a84-4b5c-b0e6-bdc1e2b9a33c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:47.542897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9526d9a-04c3-4997-98c5-5477f4006e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:47.543542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9526d9a-04c3-4997-98c5-5477f4006e6b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:47.543829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9526d9a-04c3-4997-98c5-5477f4006e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:47.544110 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9526d9a-04c3-4997-98c5-5477f4006e6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:47.610654 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9526d9a-04c3-4997-98c5-5477f4006e6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:47.612046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9526d9a-04c3-4997-98c5-5477f4006e6b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:47.674099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:47.707831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 586/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 04:00:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 04:00:47.713788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f69e7-3bfc-4b63-b3a5-0de755683996 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:47.781659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:47.821971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f69e7-3bfc-4b63-b3a5-0de755683996 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:47.855355 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94d673-f5c2-4875-b13e-33f13311b281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:47.856208 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94d673-f5c2-4875-b13e-33f13311b281 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:47.856488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94d673-f5c2-4875-b13e-33f13311b281 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:47.856762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94d673-f5c2-4875-b13e-33f13311b281 None admin] PATH_INFO: `/v3/projects/41676a0a6eb940ab93c6b5da28f567d4` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:47.858978 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f94d673-f5c2-4875-b13e-33f13311b281 None admin] RBAC: Authorizing `identity:get_project(project_id=41676a0a6eb940ab93c6b5da28f567d4)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:47.860406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f94d673-f5c2-4875-b13e-33f13311b281 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:47.862137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 598/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 04:00:47 2022] GET /identity/v3/projects/41676a0a6eb940ab93c6b5da28f567d4 => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:47.868185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9526d9a-04c3-4997-98c5-5477f4006e6b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:47.963768 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9526d9a-04c3-4997-98c5-5477f4006e6b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:47.997449 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d00915-36a7-4dcf-8a1c-d0e82cf15c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:47.998088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d00915-36a7-4dcf-8a1c-d0e82cf15c11 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:47.998417 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d00915-36a7-4dcf-8a1c-d0e82cf15c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:47.998790 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d00915-36a7-4dcf-8a1c-d0e82cf15c11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:47.999609 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d00915-36a7-4dcf-8a1c-d0e82cf15c11 None admin] RBAC: Adding query filter params (role.id=f0f7e28cbe314921afdfb657b163436f, scope.project.id=41676a0a6eb940ab93c6b5da28f567d4, user.id=d0027c47d8ec4342ba162f994420f8cf) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:47.999925 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d00915-36a7-4dcf-8a1c-d0e82cf15c11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:48.001343 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d00915-36a7-4dcf-8a1c-d0e82cf15c11 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:48.006706 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 587/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 04:00:47 2022] GET /identity/v3/role_assignments?user.id=d0027c47d8ec4342ba162f994420f8cf&scope.project.id=41676a0a6eb940ab93c6b5da28f567d4&role.id=f0f7e28cbe314921afdfb657b163436f => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:50.037174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ef155-d812-4187-b037-4d81c067945a None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:50.037702 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ef155-d812-4187-b037-4d81c067945a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:50.037794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ef155-d812-4187-b037-4d81c067945a None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:50.039172 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 599/1186] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:00:50.044450 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5143201-089d-47e8-a057-d3ffe973c2b2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:50.044848 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5143201-089d-47e8-a057-d3ffe973c2b2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:50.045013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5143201-089d-47e8-a057-d3ffe973c2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:50.062633 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f5143201-089d-47e8-a057-d3ffe973c2b2 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:50.063833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f5143201-089d-47e8-a057-d3ffe973c2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:50.089966 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:50.130604 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 588/1187] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:00:50.153742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ef155-d812-4187-b037-4d81c067945a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:50.252769 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ef155-d812-4187-b037-4d81c067945a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:50.285341 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbecf3a-dfd0-453b-a434-e311805f5e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:50.286115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbecf3a-dfd0-453b-a434-e311805f5e65 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:50.286421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbecf3a-dfd0-453b-a434-e311805f5e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:50.286702 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbecf3a-dfd0-453b-a434-e311805f5e65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:50.289273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:50.296893 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbecf3a-dfd0-453b-a434-e311805f5e65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 04:00:50.304498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-8fbecf3a-dfd0-453b-a434-e311805f5e65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:00:50.304498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:00:50.304498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:00:50.304498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:00:50.304498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:00:50.304498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:00:50.304498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:00:50.304498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:00:50.304498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:00:50.304498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:00:50.304498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:00:50.304498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:00:50.304498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 04:00:50.304498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 04:00:50.304498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 04:00:50.304498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 04:00:50.305603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 04:00:50.305603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 04:00:50.305603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 04:00:50.305603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 04:00:50.305603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:00:50.305603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:00:50.305603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:00:50.305603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:00:50.305603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:00:50.305603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:00:50.305603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:00:50.305603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:00:50.305603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:00:50.305603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:00:50.305603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:00:50.305603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:00:50.306542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:00:50.306542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:00:50.306542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:00:50.306542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:00:50.306542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 04:00:50.306542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 04:00:50.306542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 04:00:50.306542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 04:00:50.306542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 04:00:50.306542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 04:00:50.306542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:00:50.306542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:00:50.306542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 600/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 04:00:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:00:50.312543 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5143201-089d-47e8-a057-d3ffe973c2b2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:50.408612 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5143201-089d-47e8-a057-d3ffe973c2b2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:50.441886 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea527cb7-d587-4759-98cf-978ae46ebecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:50.442569 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea527cb7-d587-4759-98cf-978ae46ebecb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:50.442853 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea527cb7-d587-4759-98cf-978ae46ebecb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:50.443128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea527cb7-d587-4759-98cf-978ae46ebecb None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:50.443866 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea527cb7-d587-4759-98cf-978ae46ebecb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:50.444173 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea527cb7-d587-4759-98cf-978ae46ebecb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:50.445545 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea527cb7-d587-4759-98cf-978ae46ebecb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:50.453488 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 589/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 04:00:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:50.459183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbecf3a-dfd0-453b-a434-e311805f5e65 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:50.555465 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbecf3a-dfd0-453b-a434-e311805f5e65 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:50.587403 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792772f4-c072-48a1-b075-9c2cdc5ec5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:50.588249 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792772f4-c072-48a1-b075-9c2cdc5ec5df None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:50.588532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792772f4-c072-48a1-b075-9c2cdc5ec5df None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:50.588808 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792772f4-c072-48a1-b075-9c2cdc5ec5df None admin] PATH_INFO: `/v3/users/d0027c47d8ec4342ba162f994420f8cf` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:50.589978 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792772f4-c072-48a1-b075-9c2cdc5ec5df None admin] RBAC: Authorizing `identity:get_user(user_id=d0027c47d8ec4342ba162f994420f8cf)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:50.591438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792772f4-c072-48a1-b075-9c2cdc5ec5df None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:50.593038 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 601/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 04:00:50 2022] GET /identity/v3/users/d0027c47d8ec4342ba162f994420f8cf => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:50.598789 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea527cb7-d587-4759-98cf-978ae46ebecb None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:50.624604 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:50.695389 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea527cb7-d587-4759-98cf-978ae46ebecb None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:50.726840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7b1e5-9ea3-4203-ad95-83ad71c928fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:50.727686 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7b1e5-9ea3-4203-ad95-83ad71c928fd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:50.727985 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7b1e5-9ea3-4203-ad95-83ad71c928fd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:50.728259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7b1e5-9ea3-4203-ad95-83ad71c928fd None admin] PATH_INFO: `/v3/projects/41676a0a6eb940ab93c6b5da28f567d4` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:50.729414 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7b1e5-9ea3-4203-ad95-83ad71c928fd None admin] RBAC: Authorizing `identity:get_project(project_id=41676a0a6eb940ab93c6b5da28f567d4)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:50.730861 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7b1e5-9ea3-4203-ad95-83ad71c928fd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:50.732507 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 590/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 04:00:50 2022] GET /identity/v3/projects/41676a0a6eb940ab93c6b5da28f567d4 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:50.739061 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792772f4-c072-48a1-b075-9c2cdc5ec5df None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:50.800314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:50.837393 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792772f4-c072-48a1-b075-9c2cdc5ec5df None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:50.870912 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733e5c5-f649-4c4d-8343-43bec0ad93a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:50.871791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733e5c5-f649-4c4d-8343-43bec0ad93a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:50.872075 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733e5c5-f649-4c4d-8343-43bec0ad93a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:50.872351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733e5c5-f649-4c4d-8343-43bec0ad93a3 None admin] PATH_INFO: `/v3/projects/41676a0a6eb940ab93c6b5da28f567d4/users/d0027c47d8ec4342ba162f994420f8cf/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:50.874400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3733e5c5-f649-4c4d-8343-43bec0ad93a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=41676a0a6eb940ab93c6b5da28f567d4, user_id=d0027c47d8ec4342ba162f994420f8cf, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:50.875842 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3733e5c5-f649-4c4d-8343-43bec0ad93a3 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:50.947485 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 602/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 04:00:50 2022] PUT /identity/v3/projects/41676a0a6eb940ab93c6b5da28f567d4/users/d0027c47d8ec4342ba162f994420f8cf/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:00:52.934865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111df6b5-467b-4477-b96e-811dd60705ba None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:52.935400 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111df6b5-467b-4477-b96e-811dd60705ba None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:52.935400 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111df6b5-467b-4477-b96e-811dd60705ba None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:52.936802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 591/1193] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:00:52.942705 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67176701-6693-4860-a0ff-b49981b3963f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:52.943135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67176701-6693-4860-a0ff-b49981b3963f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:52.943271 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67176701-6693-4860-a0ff-b49981b3963f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:52.959641 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-67176701-6693-4860-a0ff-b49981b3963f None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:00:52.960828 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-67176701-6693-4860-a0ff-b49981b3963f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:00:53.024675 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 603/1194] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:00:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:00:53.049504 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111df6b5-467b-4477-b96e-811dd60705ba None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:53.151849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:53.161718 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111df6b5-467b-4477-b96e-811dd60705ba None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:53.195200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e63d5-36fd-4933-b80f-07b42c1d534a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:53.195916 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e63d5-36fd-4933-b80f-07b42c1d534a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:53.196201 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e63d5-36fd-4933-b80f-07b42c1d534a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:53.196481 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e63d5-36fd-4933-b80f-07b42c1d534a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:53.206337 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5e63d5-36fd-4933-b80f-07b42c1d534a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 04:00:53.213910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-1e5e63d5-36fd-4933-b80f-07b42c1d534a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:00:53.213910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:00:53.213910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:00:53.213910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:00:53.213910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:00:53.213910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:00:53.213910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:00:53.213910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:00:53.213910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:00:53.213910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:00:53.213910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:00:53.213910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:00:53.213910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 04:00:53.213910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 04:00:53.213910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 04:00:53.213910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 04:00:53.214995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 04:00:53.214995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 04:00:53.214995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 04:00:53.214995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 04:00:53.214995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:00:53.214995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:00:53.214995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:00:53.214995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:00:53.214995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:00:53.214995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:00:53.214995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:00:53.214995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:00:53.214995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:00:53.214995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:00:53.214995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:00:53.214995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:00:53.216021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:00:53.216021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:00:53.216021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:00:53.216021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:00:53.216021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 04:00:53.216021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 04:00:53.216021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 04:00:53.216021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 04:00:53.216021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 04:00:53.216021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 04:00:53.216021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:00:53.216021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:00:53.216021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 592/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 04:00:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:00:53.221716 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67176701-6693-4860-a0ff-b49981b3963f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:53.274781 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:53.319849 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67176701-6693-4860-a0ff-b49981b3963f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:53.352689 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486125a-46e9-4b16-b3cd-aad3ba2d3163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:53.353347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486125a-46e9-4b16-b3cd-aad3ba2d3163 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:53.353632 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486125a-46e9-4b16-b3cd-aad3ba2d3163 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:53.353911 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486125a-46e9-4b16-b3cd-aad3ba2d3163 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:53.354686 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486125a-46e9-4b16-b3cd-aad3ba2d3163 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:53.355003 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486125a-46e9-4b16-b3cd-aad3ba2d3163 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:53.356367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486125a-46e9-4b16-b3cd-aad3ba2d3163 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:53.364175 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 604/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 04:00:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:53.370211 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e63d5-36fd-4933-b80f-07b42c1d534a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:53.471477 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e63d5-36fd-4933-b80f-07b42c1d534a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:53.505046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85dca4-ff74-41a8-adb1-be28e913c1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:53.505902 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85dca4-ff74-41a8-adb1-be28e913c1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:53.506187 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85dca4-ff74-41a8-adb1-be28e913c1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:53.506490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85dca4-ff74-41a8-adb1-be28e913c1d4 None admin] PATH_INFO: `/v3/users/d0027c47d8ec4342ba162f994420f8cf` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:53.507672 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c85dca4-ff74-41a8-adb1-be28e913c1d4 None admin] RBAC: Authorizing `identity:get_user(user_id=d0027c47d8ec4342ba162f994420f8cf)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:53.509102 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c85dca4-ff74-41a8-adb1-be28e913c1d4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:53.510762 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 593/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Jun 24 04:00:53 2022] GET /identity/v3/users/d0027c47d8ec4342ba162f994420f8cf => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:53.516892 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486125a-46e9-4b16-b3cd-aad3ba2d3163 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:53.612791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486125a-46e9-4b16-b3cd-aad3ba2d3163 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:53.628688 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:53.645430 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea634b9-0fea-4997-ad04-d8017133bce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:53.646064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea634b9-0fea-4997-ad04-d8017133bce6 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:53.646347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea634b9-0fea-4997-ad04-d8017133bce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:53.646739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea634b9-0fea-4997-ad04-d8017133bce6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:53.715707 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea634b9-0fea-4997-ad04-d8017133bce6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:53.717814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea634b9-0fea-4997-ad04-d8017133bce6 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:53.812927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 605/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 04:00:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 04:00:53.818697 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85dca4-ff74-41a8-adb1-be28e913c1d4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:53.840702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:53.916011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85dca4-ff74-41a8-adb1-be28e913c1d4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:53.948296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79546311-349f-4821-8e4b-e561e9c4e1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:53.949141 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79546311-349f-4821-8e4b-e561e9c4e1ae None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:53.949426 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79546311-349f-4821-8e4b-e561e9c4e1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:53.949704 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79546311-349f-4821-8e4b-e561e9c4e1ae None admin] PATH_INFO: `/v3/projects/41676a0a6eb940ab93c6b5da28f567d4` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:53.950908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79546311-349f-4821-8e4b-e561e9c4e1ae None admin] RBAC: Authorizing `identity:get_project(project_id=41676a0a6eb940ab93c6b5da28f567d4)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:53.952885 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79546311-349f-4821-8e4b-e561e9c4e1ae None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:53.954585 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 594/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Jun 24 04:00:53 2022] GET /identity/v3/projects/41676a0a6eb940ab93c6b5da28f567d4 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:00:53.960542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea634b9-0fea-4997-ad04-d8017133bce6 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:00:53.977856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:00:54.058360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea634b9-0fea-4997-ad04-d8017133bce6 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:00:54.092524 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c453795-6dbd-4769-bcca-fbfb3edbc485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:00:54.093114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c453795-6dbd-4769-bcca-fbfb3edbc485 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:00:54.093397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c453795-6dbd-4769-bcca-fbfb3edbc485 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:00:54.093673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c453795-6dbd-4769-bcca-fbfb3edbc485 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:00:54.094523 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c453795-6dbd-4769-bcca-fbfb3edbc485 None admin] RBAC: Adding query filter params (role.id=f0f7e28cbe314921afdfb657b163436f, scope.project.id=41676a0a6eb940ab93c6b5da28f567d4, user.id=d0027c47d8ec4342ba162f994420f8cf) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:00:54.094835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c453795-6dbd-4769-bcca-fbfb3edbc485 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:00:54.096262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c453795-6dbd-4769-bcca-fbfb3edbc485 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:00:54.101831 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 606/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 04:00:53 2022] GET /identity/v3/role_assignments?user.id=d0027c47d8ec4342ba162f994420f8cf&scope.project.id=41676a0a6eb940ab93c6b5da28f567d4&role.id=f0f7e28cbe314921afdfb657b163436f => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:01:03.944273 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=56 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:01:04.099314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=60 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:08.424968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8743ca1-360c-4aec-bd61-64cbab7c7e86 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:08.425549 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8743ca1-360c-4aec-bd61-64cbab7c7e86 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:08.425549 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8743ca1-360c-4aec-bd61-64cbab7c7e86 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:08.426944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 595/1201] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:02:08.432039 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed7f5f-59f0-445b-ad89-bdcd8449e523 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:08.432432 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed7f5f-59f0-445b-ad89-bdcd8449e523 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:08.432626 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed7f5f-59f0-445b-ad89-bdcd8449e523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:08.449440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-15ed7f5f-59f0-445b-ad89-bdcd8449e523 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:02:08.450634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-15ed7f5f-59f0-445b-ad89-bdcd8449e523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:02:08.517663 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 607/1202] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:02:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:02:08.542811 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8743ca1-360c-4aec-bd61-64cbab7c7e86 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:08.645076 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8743ca1-360c-4aec-bd61-64cbab7c7e86 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:08.678241 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fd8ff-c4bd-4a10-9dcd-5a8632b31ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:08.678993 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fd8ff-c4bd-4a10-9dcd-5a8632b31ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:08.679275 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fd8ff-c4bd-4a10-9dcd-5a8632b31ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:08.679552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fd8ff-c4bd-4a10-9dcd-5a8632b31ee7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:08.690919 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21fd8ff-c4bd-4a10-9dcd-5a8632b31ee7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:08.692413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21fd8ff-c4bd-4a10-9dcd-5a8632b31ee7 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:08.709763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-b21fd8ff-c4bd-4a10-9dcd-5a8632b31ee7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:08.709763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:08.709763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:08.709763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:08.709763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:08.709763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:08.709763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:08.709763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:08.709763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:08.709763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:08.709763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:08.709763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:08.709763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 04:02:08.709763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 04:02:08.709763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 04:02:08.710956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 04:02:08.710956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:08.710956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:08.710956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:08.710956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:08.710956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:08.710956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:08.710956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:08.710956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:08.710956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:08.710956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:08.710956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:08.710956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:08.710956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:08.710956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:08.710956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:08.710956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:08.711982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 04:02:08.711982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 04:02:08.711982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:08.711982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:02:08.711982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 596/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:02:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:08.717207 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed7f5f-59f0-445b-ad89-bdcd8449e523 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:08.814944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed7f5f-59f0-445b-ad89-bdcd8449e523 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:08.843603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:08.849098 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89cd465-9151-4c33-b529-72ddfc7d2678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:08.849751 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cd465-9151-4c33-b529-72ddfc7d2678 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:08.850034 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cd465-9151-4c33-b529-72ddfc7d2678 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:08.850333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cd465-9151-4c33-b529-72ddfc7d2678 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:08.851084 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89cd465-9151-4c33-b529-72ddfc7d2678 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:08.851431 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89cd465-9151-4c33-b529-72ddfc7d2678 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:08.852783 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89cd465-9151-4c33-b529-72ddfc7d2678 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:08.864083 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 608/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 04:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:08.869791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fd8ff-c4bd-4a10-9dcd-5a8632b31ee7 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:08.922085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:08.967756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fd8ff-c4bd-4a10-9dcd-5a8632b31ee7 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:09.000976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b417e839-e5fb-43ba-8504-07d60670d519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:09.001870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b417e839-e5fb-43ba-8504-07d60670d519 None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:09.002674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b417e839-e5fb-43ba-8504-07d60670d519 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:09.002955 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b417e839-e5fb-43ba-8504-07d60670d519 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:09.003702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b417e839-e5fb-43ba-8504-07d60670d519 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:09.005137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b417e839-e5fb-43ba-8504-07d60670d519 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:09.138787 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 597/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Jun 24 04:02:08 2022] POST /identity/v3/users => generated 251 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:02:11.219593 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020699f-d11c-4deb-8954-a41e1525b1cb None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:11.220140 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020699f-d11c-4deb-8954-a41e1525b1cb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:11.220140 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020699f-d11c-4deb-8954-a41e1525b1cb None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:11.221541 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 609/1206] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:02:11.226749 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ca99c-b1ba-4b74-9162-7ef006d673d7 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:11.227144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ca99c-b1ba-4b74-9162-7ef006d673d7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:11.229818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ca99c-b1ba-4b74-9162-7ef006d673d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:11.243941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-671ca99c-b1ba-4b74-9162-7ef006d673d7 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:02:11.245125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-671ca99c-b1ba-4b74-9162-7ef006d673d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:02:11.311880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 598/1207] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:02:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:02:11.335438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020699f-d11c-4deb-8954-a41e1525b1cb None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:11.439600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020699f-d11c-4deb-8954-a41e1525b1cb None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:11.473029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9d12a-a986-45a4-8e59-4e935fbc6f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:11.473824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9d12a-a986-45a4-8e59-4e935fbc6f6f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:11.474157 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9d12a-a986-45a4-8e59-4e935fbc6f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:11.474467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9d12a-a986-45a4-8e59-4e935fbc6f6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:11.484983 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb9d12a-a986-45a4-8e59-4e935fbc6f6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 04:02:11.492794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-4eb9d12a-a986-45a4-8e59-4e935fbc6f6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 04:02:11.492794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:11.492794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:11.492794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:11.492794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:11.492794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:11.492794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:11.492794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:11.492794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:11.492794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:11.492794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:11.492794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:11.492794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 04:02:11.492794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 04:02:11.492794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 04:02:11.492794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 04:02:11.493952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 04:02:11.493952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 04:02:11.493952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 04:02:11.493952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 04:02:11.493952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:11.493952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:11.493952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:11.493952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:11.493952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:11.493952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:11.493952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:11.493952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:11.493952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:11.493952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:11.493952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:11.493952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:11.500648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:11.500648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:11.500648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:11.500648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:11.500648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 04:02:11.500648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 04:02:11.500648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 04:02:11.500648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 04:02:11.500648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 04:02:11.500648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 04:02:11.500648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 04:02:11.500648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:11.500648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 610/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 04:02:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:11.501548 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ca99c-b1ba-4b74-9162-7ef006d673d7 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:11.539696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:11.597756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ca99c-b1ba-4b74-9162-7ef006d673d7 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:11.632715 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ac3db-5116-4a6d-bd9c-d2cac4af0333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:11.633377 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ac3db-5116-4a6d-bd9c-d2cac4af0333 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:11.633661 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ac3db-5116-4a6d-bd9c-d2cac4af0333 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:11.633938 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ac3db-5116-4a6d-bd9c-d2cac4af0333 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:11.634718 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ac3db-5116-4a6d-bd9c-d2cac4af0333 None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:11.635026 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ac3db-5116-4a6d-bd9c-d2cac4af0333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:11.636427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3ac3db-5116-4a6d-bd9c-d2cac4af0333 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:11.644563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 599/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 04:02:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:02:11.650476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9d12a-a986-45a4-8e59-4e935fbc6f6f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:11.700140 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:11.749058 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9d12a-a986-45a4-8e59-4e935fbc6f6f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:11.781643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb9792-7a01-45f9-92b8-7293b939f713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:11.782453 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb9792-7a01-45f9-92b8-7293b939f713 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:11.782737 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb9792-7a01-45f9-92b8-7293b939f713 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:11.783009 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb9792-7a01-45f9-92b8-7293b939f713 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:11.793357 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb9792-7a01-45f9-92b8-7293b939f713 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:11.794842 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb9792-7a01-45f9-92b8-7293b939f713 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:11.812285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-78eb9792-7a01-45f9-92b8-7293b939f713 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:11.812285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:11.812285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:11.812285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:11.812285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:11.812285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:11.812285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:11.812285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:11.812285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:11.812285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:11.812285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:11.812285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:11.812285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 04:02:11.812285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 04:02:11.812285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 04:02:11.813255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 04:02:11.813255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:11.813255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:11.813255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:11.813255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:11.813255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:11.813255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:11.813255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:11.813255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:11.813255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:11.813255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:11.813255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:11.813255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:11.813255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:11.813255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:11.813255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:11.813255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:11.814265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 04:02:11.814265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 04:02:11.814265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:11.814265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:11.814265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 611/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:02:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:11.819441 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ac3db-5116-4a6d-bd9c-d2cac4af0333 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:11.914465 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:11.917337 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ac3db-5116-4a6d-bd9c-d2cac4af0333 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:11.949807 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae1f45-e62d-4d06-b7fc-b1b3563c318b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:11.951070 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae1f45-e62d-4d06-b7fc-b1b3563c318b None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:11.951070 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae1f45-e62d-4d06-b7fc-b1b3563c318b None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:11.951263 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae1f45-e62d-4d06-b7fc-b1b3563c318b None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:11.952573 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae1f45-e62d-4d06-b7fc-b1b3563c318b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:11.952891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae1f45-e62d-4d06-b7fc-b1b3563c318b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:11.954283 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae1f45-e62d-4d06-b7fc-b1b3563c318b None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:11.966409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 600/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 04:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:11.972674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb9792-7a01-45f9-92b8-7293b939f713 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:12.073823 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb9792-7a01-45f9-92b8-7293b939f713 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:12.075425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:12.109095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2a88c-c8f7-4777-9aca-606e71a29d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:12.109958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2a88c-c8f7-4777-9aca-606e71a29d68 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:12.110241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2a88c-c8f7-4777-9aca-606e71a29d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:12.110691 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2a88c-c8f7-4777-9aca-606e71a29d68 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:12.117988 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a2a88c-c8f7-4777-9aca-606e71a29d68 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:12.119445 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a2a88c-c8f7-4777-9aca-606e71a29d68 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:12.134581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-f4a2a88c-c8f7-4777-9aca-606e71a29d68 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 04:02:12.134581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:12.134581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:12.134581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:12.134581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:12.134581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:12.134581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:12.134581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:12.134581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:12.134581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:12.134581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:12.134581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:12.134581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 04:02:12.134581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 04:02:12.134581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 04:02:12.134581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 04:02:12.135787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:12.135787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:12.135787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 04:02:12.135787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 04:02:12.135787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 04:02:12.135787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 04:02:12.135787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:12.135787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:12.135787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:12.135787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:12.135787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:12.135787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:12.135787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:12.135787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:12.135787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:12.135787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:12.135787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:12.135787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:12.137001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:12.137001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:12.137001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 04:02:12.137001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 04:02:12.137001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 04:02:12.137001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 04:02:12.137001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 04:02:12.137001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 04:02:12.137001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 04:02:12.137001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:12.137001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 612/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:02:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:12.142842 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae1f45-e62d-4d06-b7fc-b1b3563c318b None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:12.243645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae1f45-e62d-4d06-b7fc-b1b3563c318b None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:12.277879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc02257-c26b-4831-a7c2-5a60dac0a86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:12.278566 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc02257-c26b-4831-a7c2-5a60dac0a86e None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:12.278841 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc02257-c26b-4831-a7c2-5a60dac0a86e None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:12.279118 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc02257-c26b-4831-a7c2-5a60dac0a86e None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:12.280620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc02257-c26b-4831-a7c2-5a60dac0a86e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:12.280926 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc02257-c26b-4831-a7c2-5a60dac0a86e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:12.282473 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc02257-c26b-4831-a7c2-5a60dac0a86e None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:12.299078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 601/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Fri Jun 24 04:02:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:12.304852 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2a88c-c8f7-4777-9aca-606e71a29d68 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:12.402955 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2a88c-c8f7-4777-9aca-606e71a29d68 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:12.437831 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40daeee-1740-47b3-9543-991ff6d63119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:12.438503 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40daeee-1740-47b3-9543-991ff6d63119 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:12.438791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40daeee-1740-47b3-9543-991ff6d63119 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:12.439067 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40daeee-1740-47b3-9543-991ff6d63119 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:12.468837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:12.505261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40daeee-1740-47b3-9543-991ff6d63119 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:12.506685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40daeee-1740-47b3-9543-991ff6d63119 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:12.602326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 613/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 04:02:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 04:02:12.608208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc02257-c26b-4831-a7c2-5a60dac0a86e None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:12.627613 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:12.711625 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc02257-c26b-4831-a7c2-5a60dac0a86e None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:12.743815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d10f27-4239-4cdd-b376-2e9d77a5f77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:12.744593 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d10f27-4239-4cdd-b376-2e9d77a5f77c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:12.744873 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d10f27-4239-4cdd-b376-2e9d77a5f77c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:12.745149 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d10f27-4239-4cdd-b376-2e9d77a5f77c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:12.760086 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d10f27-4239-4cdd-b376-2e9d77a5f77c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:12.760928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d10f27-4239-4cdd-b376-2e9d77a5f77c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:12.779325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-d3d10f27-4239-4cdd-b376-2e9d77a5f77c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:12.779325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:12.779325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:12.779325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:12.779325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:12.779325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:12.779325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:12.779325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:12.779325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:12.779325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:12.779325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:12.779325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:12.779325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 04:02:12.779325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 04:02:12.779325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 04:02:12.783530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 04:02:12.783530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:12.783530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:12.783530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:12.783530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:12.783530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:12.783530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:12.783530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:12.783530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:12.783530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:12.783530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:12.783530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:12.783530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:12.783530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:12.783530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:12.783530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:12.783530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:12.784789 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 04:02:12.784789 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 04:02:12.784789 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:12.784789 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:02:12.784789 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 602/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:12.786428 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40daeee-1740-47b3-9543-991ff6d63119 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:12.845896 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:12.885359 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40daeee-1740-47b3-9543-991ff6d63119 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:12.917740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5c5af-8a15-4f42-8339-08678b0e1b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:12.918418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5c5af-8a15-4f42-8339-08678b0e1b00 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:12.918710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5c5af-8a15-4f42-8339-08678b0e1b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:12.918989 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5c5af-8a15-4f42-8339-08678b0e1b00 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:12.919708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5c5af-8a15-4f42-8339-08678b0e1b00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:12.920016 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5c5af-8a15-4f42-8339-08678b0e1b00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:12.921972 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5c5af-8a15-4f42-8339-08678b0e1b00 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:12.933596 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 614/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 04:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:12.944385 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d10f27-4239-4cdd-b376-2e9d77a5f77c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:13.026721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:13.058641 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d10f27-4239-4cdd-b376-2e9d77a5f77c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:13.093061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13289606-076e-441b-bf2a-0bf266981bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:13.093853 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13289606-076e-441b-bf2a-0bf266981bfe None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:13.094143 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13289606-076e-441b-bf2a-0bf266981bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:13.094444 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13289606-076e-441b-bf2a-0bf266981bfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:13.101061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13289606-076e-441b-bf2a-0bf266981bfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:13.106313 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13289606-076e-441b-bf2a-0bf266981bfe None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:13.116572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-13289606-076e-441b-bf2a-0bf266981bfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 04:02:13.116572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:13.116572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:13.116572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:13.116572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:13.116572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:13.116572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:13.116572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:13.116572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:13.116572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:13.116572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:13.116572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:13.116572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 04:02:13.116572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 04:02:13.116572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 04:02:13.117639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 04:02:13.117639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:13.117639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:13.117639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:13.117639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:13.117639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:13.117639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:13.117639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:13.117639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:13.117639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:13.117639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:13.117639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:13.117639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:13.117639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:13.117639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:13.117639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:13.117639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:13.118952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 04:02:13.118952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 04:02:13.118952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 04:02:13.118952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 04:02:13.118952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 04:02:13.118952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 04:02:13.118952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 04:02:13.118952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:02:13.118952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 603/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 04:02:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:13.124581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5c5af-8a15-4f42-8339-08678b0e1b00 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:13.222743 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5c5af-8a15-4f42-8339-08678b0e1b00 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:13.246937 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:13.257951 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca55453-abf9-4881-ad2f-18281b0b1a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:13.258642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca55453-abf9-4881-ad2f-18281b0b1a4b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:13.258931 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca55453-abf9-4881-ad2f-18281b0b1a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:13.259257 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca55453-abf9-4881-ad2f-18281b0b1a4b None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:13.260043 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca55453-abf9-4881-ad2f-18281b0b1a4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:13.260352 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca55453-abf9-4881-ad2f-18281b0b1a4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:13.261777 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca55453-abf9-4881-ad2f-18281b0b1a4b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:13.273904 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 615/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 04:02:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:13.282939 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13289606-076e-441b-bf2a-0bf266981bfe None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:13.386670 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13289606-076e-441b-bf2a-0bf266981bfe None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:13.414960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:13.421225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd91cc0-f746-4aa7-b484-001c7b1de270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:13.421910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd91cc0-f746-4aa7-b484-001c7b1de270 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:13.422196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd91cc0-f746-4aa7-b484-001c7b1de270 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:13.422503 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd91cc0-f746-4aa7-b484-001c7b1de270 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:13.423322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd91cc0-f746-4aa7-b484-001c7b1de270 None admin] RBAC: Adding query filter params (role.id=3073b5cac26f43089c9ed8c6a5102cff, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=7662e61619d44c92af3ffacbacba3913) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:13.423636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd91cc0-f746-4aa7-b484-001c7b1de270 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:13.425147 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd91cc0-f746-4aa7-b484-001c7b1de270 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:13.430599 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 604/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 04:02:13 2022] GET /identity/v3/role_assignments?user.id=7662e61619d44c92af3ffacbacba3913&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=3073b5cac26f43089c9ed8c6a5102cff => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:15.456480 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde053e-6da0-40bf-bd39-5b64323f8078 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:15.457013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde053e-6da0-40bf-bd39-5b64323f8078 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:15.457127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccde053e-6da0-40bf-bd39-5b64323f8078 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:15.458437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 616/1220] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:02:15.463690 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc9d53-0f98-4f82-8634-33af6f78d41c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:15.464064 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc9d53-0f98-4f82-8634-33af6f78d41c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:15.464258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc9d53-0f98-4f82-8634-33af6f78d41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:15.480614 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-21cc9d53-0f98-4f82-8634-33af6f78d41c None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:02:15.481797 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-21cc9d53-0f98-4f82-8634-33af6f78d41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:02:15.548176 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 605/1221] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:02:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:02:15.571328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde053e-6da0-40bf-bd39-5b64323f8078 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:15.671582 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccde053e-6da0-40bf-bd39-5b64323f8078 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:15.705481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccd218-e5e1-4a87-aa50-91c1847fb945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:15.706275 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccd218-e5e1-4a87-aa50-91c1847fb945 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:15.706632 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccd218-e5e1-4a87-aa50-91c1847fb945 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:15.706866 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caccd218-e5e1-4a87-aa50-91c1847fb945 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:15.716997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caccd218-e5e1-4a87-aa50-91c1847fb945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 04:02:15.724748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-caccd218-e5e1-4a87-aa50-91c1847fb945 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 04:02:15.724748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:15.724748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:15.724748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:15.724748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:15.724748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:15.724748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:15.724748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:15.724748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:15.724748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:15.724748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:15.724748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:15.724748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 04:02:15.724748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 04:02:15.724748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 04:02:15.724748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 04:02:15.725827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 04:02:15.725827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 04:02:15.725827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 04:02:15.725827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 04:02:15.725827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:15.725827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:15.725827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:15.725827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:15.725827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:15.725827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:15.725827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:15.725827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:15.725827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:15.725827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:15.725827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:15.725827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:15.726790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:15.726790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:15.726790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:15.726790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:15.726790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 04:02:15.726790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 04:02:15.726790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 04:02:15.726790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 04:02:15.726790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 04:02:15.726790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 04:02:15.726790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 04:02:15.726790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:15.726790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 617/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 04:02:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:15.732433 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc9d53-0f98-4f82-8634-33af6f78d41c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:15.831675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc9d53-0f98-4f82-8634-33af6f78d41c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:15.854678 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:15.865495 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a124795-46e9-4bec-9951-eb483db24e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:15.866158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a124795-46e9-4bec-9951-eb483db24e0c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:15.866475 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a124795-46e9-4bec-9951-eb483db24e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:15.866757 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a124795-46e9-4bec-9951-eb483db24e0c None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:15.867502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a124795-46e9-4bec-9951-eb483db24e0c None admin] RBAC: Adding query filter params (name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:15.867813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a124795-46e9-4bec-9951-eb483db24e0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:15.869177 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a124795-46e9-4bec-9951-eb483db24e0c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:15.877703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 606/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 04:02:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:15.883711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccd218-e5e1-4a87-aa50-91c1847fb945 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:15.926739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:15.982740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caccd218-e5e1-4a87-aa50-91c1847fb945 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:16.016877 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d02bd-d70e-4487-9e39-8fbfb5f8680f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:16.017660 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d02bd-d70e-4487-9e39-8fbfb5f8680f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:16.017941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d02bd-d70e-4487-9e39-8fbfb5f8680f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:16.018218 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d02bd-d70e-4487-9e39-8fbfb5f8680f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:16.028390 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d02bd-d70e-4487-9e39-8fbfb5f8680f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:16.029835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d02bd-d70e-4487-9e39-8fbfb5f8680f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:16.048491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-bb1d02bd-d70e-4487-9e39-8fbfb5f8680f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:16.048491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:16.048491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:16.048491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:16.048491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:16.048491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:16.048491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:16.048491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:16.048491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:16.048491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:16.048491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:16.048491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:16.048491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 04:02:16.048491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 04:02:16.048491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 04:02:16.049561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 04:02:16.049561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:16.049561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:16.049561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:16.049561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:16.049561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:16.049561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:16.049561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:16.049561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:16.049561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:16.049561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:16.049561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:16.049561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:16.049561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:16.049561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:16.049561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:16.049561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:16.050744 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 04:02:16.050744 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 04:02:16.050744 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:16.050744 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:16.050744 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 618/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:02:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:16.055576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a124795-46e9-4bec-9951-eb483db24e0c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:16.157252 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a124795-46e9-4bec-9951-eb483db24e0c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:16.191280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32acbc-e771-45f1-bd51-6b4a200cf821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:16.191935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32acbc-e771-45f1-bd51-6b4a200cf821 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:16.192223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32acbc-e771-45f1-bd51-6b4a200cf821 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:16.192495 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32acbc-e771-45f1-bd51-6b4a200cf821 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:16.193200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32acbc-e771-45f1-bd51-6b4a200cf821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:16.193504 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32acbc-e771-45f1-bd51-6b4a200cf821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:16.194889 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32acbc-e771-45f1-bd51-6b4a200cf821 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:16.206870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 607/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 04:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:16.212736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d02bd-d70e-4487-9e39-8fbfb5f8680f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:16.307041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:16.313886 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d02bd-d70e-4487-9e39-8fbfb5f8680f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:16.348118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb89af-ab9d-4645-9123-716bf7664596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:16.348952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb89af-ab9d-4645-9123-716bf7664596 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:16.349242 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb89af-ab9d-4645-9123-716bf7664596 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:16.349520 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb89af-ab9d-4645-9123-716bf7664596 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:16.356585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb89af-ab9d-4645-9123-716bf7664596 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:16.358035 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb89af-ab9d-4645-9123-716bf7664596 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:16.373539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-eefb89af-ab9d-4645-9123-716bf7664596 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 04:02:16.373539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:16.373539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:16.373539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:16.373539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:16.373539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:16.373539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:16.373539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:16.373539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:16.373539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:16.373539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:16.373539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:16.373539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 04:02:16.373539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 04:02:16.373539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 04:02:16.373539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 04:02:16.374570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:16.374570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:16.374570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 04:02:16.374570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 04:02:16.374570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 04:02:16.374570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 04:02:16.374570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:16.374570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:16.374570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:16.374570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:16.374570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:16.374570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:16.374570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:16.374570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:16.374570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:16.374570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:16.374570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:16.374570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:16.375739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:16.375739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:16.375739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 04:02:16.375739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 04:02:16.375739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 04:02:16.375739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 04:02:16.375739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 04:02:16.375739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 04:02:16.375739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 04:02:16.375739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:16.375739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 619/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:02:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:16.380719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32acbc-e771-45f1-bd51-6b4a200cf821 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:16.428487 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:16.480703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32acbc-e771-45f1-bd51-6b4a200cf821 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:16.514048 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ba462-b908-441d-b89c-383d19007570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:16.514740 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ba462-b908-441d-b89c-383d19007570 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:16.515027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ba462-b908-441d-b89c-383d19007570 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:16.515979 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ba462-b908-441d-b89c-383d19007570 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:16.516899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ba462-b908-441d-b89c-383d19007570 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:16.517210 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ba462-b908-441d-b89c-383d19007570 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:16.518671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7ba462-b908-441d-b89c-383d19007570 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:16.536725 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 608/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Fri Jun 24 04:02:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:16.542536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb89af-ab9d-4645-9123-716bf7664596 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:16.640700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb89af-ab9d-4645-9123-716bf7664596 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:16.675321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4fd94-ae73-449a-b024-dbdce790792a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:16.676101 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4fd94-ae73-449a-b024-dbdce790792a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:16.676383 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4fd94-ae73-449a-b024-dbdce790792a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:16.676658 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c4fd94-ae73-449a-b024-dbdce790792a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:16.686301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4fd94-ae73-449a-b024-dbdce790792a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:16.687767 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c4fd94-ae73-449a-b024-dbdce790792a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:16.705373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-28c4fd94-ae73-449a-b024-dbdce790792a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:16.705373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:16.705373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:16.705373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:16.705373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:16.705373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:16.705373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:16.705373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:16.705373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:16.705373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:16.705373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:16.705373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:16.705373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 04:02:16.705373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 04:02:16.705373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 04:02:16.706481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 04:02:16.706481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:16.706481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:16.706481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:16.706481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:16.706481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:16.706481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:16.706481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:16.706481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:16.706481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:16.706481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:16.706481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:16.706481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:16.706481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:16.706481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:16.706481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:16.706481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:16.707671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 04:02:16.707671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 04:02:16.707671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:16.707671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:16.707671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 620/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:16.712210 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ba462-b908-441d-b89c-383d19007570 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:16.808345 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7ba462-b908-441d-b89c-383d19007570 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:16.810671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:16.841533 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd7304-4d94-4cf8-a0bf-ac7620153beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:16.842184 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd7304-4d94-4cf8-a0bf-ac7620153beb None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:16.842502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd7304-4d94-4cf8-a0bf-ac7620153beb None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:16.842785 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd7304-4d94-4cf8-a0bf-ac7620153beb None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:16.843503 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd7304-4d94-4cf8-a0bf-ac7620153beb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:16.843814 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd7304-4d94-4cf8-a0bf-ac7620153beb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:16.845908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd7304-4d94-4cf8-a0bf-ac7620153beb None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:16.856983 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 609/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 04:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:16.863068 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4fd94-ae73-449a-b024-dbdce790792a None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:16.872046 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:16.961333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c4fd94-ae73-449a-b024-dbdce790792a None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:16.994143 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33630807-131a-4521-9932-4ba1b74e1dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:16.994951 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33630807-131a-4521-9932-4ba1b74e1dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:16.995237 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33630807-131a-4521-9932-4ba1b74e1dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:16.995768 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33630807-131a-4521-9932-4ba1b74e1dd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:17.003378 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33630807-131a-4521-9932-4ba1b74e1dd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:17.004809 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33630807-131a-4521-9932-4ba1b74e1dd5 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:17.019775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-33630807-131a-4521-9932-4ba1b74e1dd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 04:02:17.019775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:17.019775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:17.019775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:17.019775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:17.019775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:17.019775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:17.019775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:17.019775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:17.019775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:17.019775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:17.019775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:17.019775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 04:02:17.019775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 04:02:17.019775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 04:02:17.020826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 04:02:17.020826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:17.020826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:17.020826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:17.020826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:17.020826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:17.020826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:17.020826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:17.020826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:17.020826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:17.020826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:17.020826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:17.020826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:17.020826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:17.020826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:17.020826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:17.020826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:17.021935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 04:02:17.021935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 04:02:17.021935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 04:02:17.021935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 04:02:17.021935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 04:02:17.021935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 04:02:17.021935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 04:02:17.021935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:17.021935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 621/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 04:02:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:17.026888 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd7304-4d94-4cf8-a0bf-ac7620153beb None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:17.128816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd7304-4d94-4cf8-a0bf-ac7620153beb None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:17.161718 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9773f-e360-4589-a320-183baebe10f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:17.162393 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9773f-e360-4589-a320-183baebe10f0 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:17.162680 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9773f-e360-4589-a320-183baebe10f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:17.162958 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9773f-e360-4589-a320-183baebe10f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:17.163743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9773f-e360-4589-a320-183baebe10f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:17.164053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9773f-e360-4589-a320-183baebe10f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:17.165484 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9773f-e360-4589-a320-183baebe10f0 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:17.178245 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 610/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 04:02:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:17.184157 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33630807-131a-4521-9932-4ba1b74e1dd5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:17.249528 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:17.283691 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33630807-131a-4521-9932-4ba1b74e1dd5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:17.317259 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a50c2-2333-4659-9b31-c57f76b080bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:17.318145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a50c2-2333-4659-9b31-c57f76b080bb None admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:17.318459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a50c2-2333-4659-9b31-c57f76b080bb None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:17.318740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a50c2-2333-4659-9b31-c57f76b080bb None admin] PATH_INFO: `/v3/projects/68d376986e854a4bb334676572340b4c/users/7662e61619d44c92af3ffacbacba3913/roles/3073b5cac26f43089c9ed8c6a5102cff` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:17.332769 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a50c2-2333-4659-9b31-c57f76b080bb None admin] RBAC: Authorizing `identity:create_grant(project_id=68d376986e854a4bb334676572340b4c, user_id=7662e61619d44c92af3ffacbacba3913, role_id=3073b5cac26f43089c9ed8c6a5102cff)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:17.334261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a50c2-2333-4659-9b31-c57f76b080bb None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:17.401351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 622/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 04:02:17 2022] PUT /identity/v3/projects/68d376986e854a4bb334676572340b4c/users/7662e61619d44c92af3ffacbacba3913/roles/3073b5cac26f43089c9ed8c6a5102cff => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:02:19.450855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6628b72-d16e-4f64-8c10-f52109639401 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:19.451429 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6628b72-d16e-4f64-8c10-f52109639401 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:19.451429 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6628b72-d16e-4f64-8c10-f52109639401 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:19.452808 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 611/1233] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:02:19.458239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105caa41-c9f6-4d22-bc20-f256c7c10335 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:19.458705 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105caa41-c9f6-4d22-bc20-f256c7c10335 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:19.458832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105caa41-c9f6-4d22-bc20-f256c7c10335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:19.476109 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-105caa41-c9f6-4d22-bc20-f256c7c10335 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:02:19.477292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-105caa41-c9f6-4d22-bc20-f256c7c10335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:02:19.543673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 623/1234] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:02:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:02:19.566974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6628b72-d16e-4f64-8c10-f52109639401 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:19.592085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:19.669502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6628b72-d16e-4f64-8c10-f52109639401 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:19.712502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce59345-acd1-456d-96b3-a8cbfe94e9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:19.713210 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce59345-acd1-456d-96b3-a8cbfe94e9ea None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:19.713491 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce59345-acd1-456d-96b3-a8cbfe94e9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:19.713791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce59345-acd1-456d-96b3-a8cbfe94e9ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:19.724845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce59345-acd1-456d-96b3-a8cbfe94e9ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 04:02:19.732431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-7ce59345-acd1-456d-96b3-a8cbfe94e9ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 04:02:19.732431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:19.732431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:19.732431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:19.732431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:19.732431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:19.732431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:19.732431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:19.732431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:19.732431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:19.732431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:19.732431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:19.732431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 04:02:19.732431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 04:02:19.732431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 04:02:19.732431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 04:02:19.733517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 04:02:19.733517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 04:02:19.733517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 04:02:19.733517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 04:02:19.733517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:19.733517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:19.733517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:19.733517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:19.733517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:19.733517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:19.733517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:19.733517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:19.733517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:19.733517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:19.733517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:19.733517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:19.734474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:19.734474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:19.734474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:19.734474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:19.734474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 04:02:19.734474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 04:02:19.734474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 04:02:19.734474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 04:02:19.734474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 04:02:19.734474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 04:02:19.734474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 04:02:19.734474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:02:19.734474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 612/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Jun 24 04:02:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:19.739648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105caa41-c9f6-4d22-bc20-f256c7c10335 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:19.789893 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:19.841390 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105caa41-c9f6-4d22-bc20-f256c7c10335 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:19.878144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472b0f5-58e5-4164-84fd-d3be7e810546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:19.878840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472b0f5-58e5-4164-84fd-d3be7e810546 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:19.879124 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472b0f5-58e5-4164-84fd-d3be7e810546 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:19.879404 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472b0f5-58e5-4164-84fd-d3be7e810546 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:19.880150 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472b0f5-58e5-4164-84fd-d3be7e810546 None admin] RBAC: Adding query filter params (name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:19.880459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472b0f5-58e5-4164-84fd-d3be7e810546 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:19.881834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472b0f5-58e5-4164-84fd-d3be7e810546 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:19.889742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 624/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Jun 24 04:02:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:19.896511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce59345-acd1-456d-96b3-a8cbfe94e9ea None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:19.980562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:19.996142 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce59345-acd1-456d-96b3-a8cbfe94e9ea None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:20.050968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77e7a1-8fb3-4c95-983b-49c316fd7c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:20.050968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77e7a1-8fb3-4c95-983b-49c316fd7c63 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:20.050968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77e7a1-8fb3-4c95-983b-49c316fd7c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:20.051345 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77e7a1-8fb3-4c95-983b-49c316fd7c63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:20.062788 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77e7a1-8fb3-4c95-983b-49c316fd7c63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:20.065255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77e7a1-8fb3-4c95-983b-49c316fd7c63 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:20.086777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-ba77e7a1-8fb3-4c95-983b-49c316fd7c63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:20.086777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:20.086777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:20.086777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:20.086777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:20.086777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:20.086777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:20.086777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:20.086777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:20.086777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:20.086777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:20.086777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:20.086777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 04:02:20.086777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 04:02:20.086777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 04:02:20.087882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 04:02:20.087882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:20.087882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:20.087882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:20.087882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:20.087882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:20.087882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:20.087882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:20.087882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:20.087882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:20.087882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:20.087882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:20.087882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:20.087882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:20.087882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:20.087882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:20.087882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:20.088957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 04:02:20.088957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 04:02:20.088957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:20.088957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:02:20.088957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 613/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:02:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:20.093806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472b0f5-58e5-4164-84fd-d3be7e810546 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:20.201754 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472b0f5-58e5-4164-84fd-d3be7e810546 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:20.210942 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:20.234682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1b90f-4d35-4c9e-8bc8-b56419798533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:20.235826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1b90f-4d35-4c9e-8bc8-b56419798533 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:20.236572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1b90f-4d35-4c9e-8bc8-b56419798533 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:20.236572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1b90f-4d35-4c9e-8bc8-b56419798533 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:20.237075 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1b90f-4d35-4c9e-8bc8-b56419798533 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:20.237385 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1b90f-4d35-4c9e-8bc8-b56419798533 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:20.238785 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e1b90f-4d35-4c9e-8bc8-b56419798533 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:20.250539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 625/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 04:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:20.256910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77e7a1-8fb3-4c95-983b-49c316fd7c63 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:20.355617 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77e7a1-8fb3-4c95-983b-49c316fd7c63 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:20.395299 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95cded-c783-413e-9b13-11aa21f4f9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:20.396103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95cded-c783-413e-9b13-11aa21f4f9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:20.396386 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95cded-c783-413e-9b13-11aa21f4f9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:20.396663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95cded-c783-413e-9b13-11aa21f4f9b1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:20.399498 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:20.404751 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95cded-c783-413e-9b13-11aa21f4f9b1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:20.406192 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95cded-c783-413e-9b13-11aa21f4f9b1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:20.420108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-cc95cded-c783-413e-9b13-11aa21f4f9b1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 04:02:20.420108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:20.420108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:20.420108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:20.420108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:20.420108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:20.420108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:20.420108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:20.420108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:20.420108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:20.420108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:20.420108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:20.420108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 04:02:20.420108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 04:02:20.420108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 04:02:20.420108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 04:02:20.421274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:20.421274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:20.421274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 04:02:20.421274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 04:02:20.421274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 04:02:20.421274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 04:02:20.421274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:20.421274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:20.421274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:20.421274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:20.421274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:20.421274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:20.421274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:20.421274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:20.421274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:20.421274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:20.421274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:20.421274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:20.423296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:20.423296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:20.423296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 04:02:20.423296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 04:02:20.423296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 04:02:20.423296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 04:02:20.423296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 04:02:20.423296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 04:02:20.423296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 04:02:20.423296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:02:20.423296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 614/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:02:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:20.427710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1b90f-4d35-4c9e-8bc8-b56419798533 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:20.526582 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1b90f-4d35-4c9e-8bc8-b56419798533 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:20.560255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a6473-c822-4bc3-821e-8126283af3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:20.560915 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a6473-c822-4bc3-821e-8126283af3cd None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:20.561199 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a6473-c822-4bc3-821e-8126283af3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:20.561473 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a6473-c822-4bc3-821e-8126283af3cd None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:20.562394 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a6473-c822-4bc3-821e-8126283af3cd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:20.562637 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a6473-c822-4bc3-821e-8126283af3cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:20.565166 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a6473-c822-4bc3-821e-8126283af3cd None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:20.582042 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 626/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Fri Jun 24 04:02:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:20.587765 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95cded-c783-413e-9b13-11aa21f4f9b1 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:20.698527 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95cded-c783-413e-9b13-11aa21f4f9b1 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:20.732720 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909ccba-2301-4118-a4c1-551e103d8b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:20.733354 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909ccba-2301-4118-a4c1-551e103d8b9c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:20.733672 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909ccba-2301-4118-a4c1-551e103d8b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:20.733951 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4909ccba-2301-4118-a4c1-551e103d8b9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:20.795821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:20.802692 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4909ccba-2301-4118-a4c1-551e103d8b9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:20.804153 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4909ccba-2301-4118-a4c1-551e103d8b9c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:20.899905 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 615/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 04:02:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 04:02:20.905692 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a6473-c822-4bc3-821e-8126283af3cd None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:20.933676 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:21.007282 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a6473-c822-4bc3-821e-8126283af3cd None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:21.040574 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1316e17-5562-470b-8c1e-c8c4be7ef965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:21.041355 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1316e17-5562-470b-8c1e-c8c4be7ef965 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:21.041635 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1316e17-5562-470b-8c1e-c8c4be7ef965 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:21.041941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1316e17-5562-470b-8c1e-c8c4be7ef965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:21.051908 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1316e17-5562-470b-8c1e-c8c4be7ef965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:21.053304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1316e17-5562-470b-8c1e-c8c4be7ef965 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:21.070301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-b1316e17-5562-470b-8c1e-c8c4be7ef965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:21.070301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:21.070301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:21.070301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:21.070301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:21.070301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:21.070301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:21.070301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:21.070301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:21.070301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:21.070301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:21.070301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:21.070301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 04:02:21.070301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 04:02:21.070301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 04:02:21.071408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 04:02:21.071408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:21.071408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:21.071408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:21.071408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:21.071408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:21.071408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:21.071408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:21.071408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:21.071408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:21.071408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:21.071408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:21.071408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:21.071408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:21.071408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:21.071408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:21.071408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:21.072530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 04:02:21.072530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 04:02:21.072530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:21.072530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:21.072530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 627/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:02:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 04:02:21.077212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909ccba-2301-4118-a4c1-551e103d8b9c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:21.170659 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:21.176549 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4909ccba-2301-4118-a4c1-551e103d8b9c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:21.210015 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6e29e-6231-48ad-bf79-c6f1c6820e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:21.210722 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6e29e-6231-48ad-bf79-c6f1c6820e72 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:21.211008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6e29e-6231-48ad-bf79-c6f1c6820e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:21.211289 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6e29e-6231-48ad-bf79-c6f1c6820e72 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:21.212011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d6e29e-6231-48ad-bf79-c6f1c6820e72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:21.212320 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d6e29e-6231-48ad-bf79-c6f1c6820e72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:21.213694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d6e29e-6231-48ad-bf79-c6f1c6820e72 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:21.225892 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 616/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 04:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:21.231586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1316e17-5562-470b-8c1e-c8c4be7ef965 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:21.308545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:21.329767 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1316e17-5562-470b-8c1e-c8c4be7ef965 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:21.362858 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc4926-7ecf-49bc-bb85-6ce23707c489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:21.363660 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc4926-7ecf-49bc-bb85-6ce23707c489 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:21.364151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc4926-7ecf-49bc-bb85-6ce23707c489 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:21.364596 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc4926-7ecf-49bc-bb85-6ce23707c489 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:21.371878 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc4926-7ecf-49bc-bb85-6ce23707c489 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:21.373328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc4926-7ecf-49bc-bb85-6ce23707c489 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:21.387513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-78fc4926-7ecf-49bc-bb85-6ce23707c489 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 04:02:21.387513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:21.387513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:21.387513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:21.387513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:21.387513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:21.387513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:21.387513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:21.387513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:21.387513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:21.387513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:21.387513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:21.387513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 04:02:21.387513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 04:02:21.387513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 04:02:21.388577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 04:02:21.388577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:21.388577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:21.388577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:21.388577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:21.388577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:21.388577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:21.388577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:21.388577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:21.388577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:21.388577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:21.388577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:21.388577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:21.388577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:21.388577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:21.388577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:21.388577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:21.389704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 04:02:21.389704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 04:02:21.389704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 04:02:21.389704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 04:02:21.389704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 04:02:21.389704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 04:02:21.389704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 04:02:21.389704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:21.389704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 628/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 04:02:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:21.395234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6e29e-6231-48ad-bf79-c6f1c6820e72 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:21.492846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6e29e-6231-48ad-bf79-c6f1c6820e72 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:21.526508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db087ca6-1171-4c49-9fc4-4eb1f860f838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:21.527160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db087ca6-1171-4c49-9fc4-4eb1f860f838 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:21.527445 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db087ca6-1171-4c49-9fc4-4eb1f860f838 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:21.527816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db087ca6-1171-4c49-9fc4-4eb1f860f838 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:21.528567 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db087ca6-1171-4c49-9fc4-4eb1f860f838 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:21.528879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db087ca6-1171-4c49-9fc4-4eb1f860f838 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:21.530314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db087ca6-1171-4c49-9fc4-4eb1f860f838 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:21.542195 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 617/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 04:02:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:21.548487 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc4926-7ecf-49bc-bb85-6ce23707c489 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:21.649779 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc4926-7ecf-49bc-bb85-6ce23707c489 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:21.681449 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538922e6-6210-4ee1-b7d8-d72ccb27a530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:21.682094 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538922e6-6210-4ee1-b7d8-d72ccb27a530 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:21.682396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538922e6-6210-4ee1-b7d8-d72ccb27a530 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:21.682685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538922e6-6210-4ee1-b7d8-d72ccb27a530 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:21.683510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538922e6-6210-4ee1-b7d8-d72ccb27a530 None admin] RBAC: Adding query filter params (role.id=3073b5cac26f43089c9ed8c6a5102cff, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=7662e61619d44c92af3ffacbacba3913) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:21.683824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538922e6-6210-4ee1-b7d8-d72ccb27a530 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:21.685245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538922e6-6210-4ee1-b7d8-d72ccb27a530 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:21.691461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 629/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 04:02:21 2022] GET /identity/v3/role_assignments?user.id=7662e61619d44c92af3ffacbacba3913&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=3073b5cac26f43089c9ed8c6a5102cff => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:23.890843 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2556d-feeb-4b57-9ba4-7ac561e1ef11 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:23.891964 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2556d-feeb-4b57-9ba4-7ac561e1ef11 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:23.891964 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2556d-feeb-4b57-9ba4-7ac561e1ef11 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:23.893443 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 618/1247] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:02:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:02:23.898630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c825ed-493d-4e56-bbd9-67edd37faafc None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:23.899212 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c825ed-493d-4e56-bbd9-67edd37faafc None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:23.899212 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c825ed-493d-4e56-bbd9-67edd37faafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:23.904199 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:23.916079 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a8c825ed-493d-4e56-bbd9-67edd37faafc None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:02:23.917257 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a8c825ed-493d-4e56-bbd9-67edd37faafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:02:23.983190 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 630/1248] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:02:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:02:24.009033 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2556d-feeb-4b57-9ba4-7ac561e1ef11 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:24.027218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:24.112978 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2556d-feeb-4b57-9ba4-7ac561e1ef11 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:24.146438 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4691da-79b6-4dc3-87a5-440c103354e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:24.147163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4691da-79b6-4dc3-87a5-440c103354e2 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:24.147451 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4691da-79b6-4dc3-87a5-440c103354e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:24.147732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4691da-79b6-4dc3-87a5-440c103354e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:24.157839 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4691da-79b6-4dc3-87a5-440c103354e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 04:02:24.165413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-7f4691da-79b6-4dc3-87a5-440c103354e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:02:24.165413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:24.165413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:24.165413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:24.165413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:24.165413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:24.165413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:24.165413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:24.165413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:24.165413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:24.165413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:24.165413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:24.165413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 04:02:24.165413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 04:02:24.165413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 04:02:24.165413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 04:02:24.171976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 04:02:24.171976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 04:02:24.171976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 04:02:24.171976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 04:02:24.171976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:24.171976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:24.171976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:24.171976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:24.171976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:24.171976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:24.171976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:24.171976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:24.171976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:24.171976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:24.171976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:24.171976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:24.173910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:24.173910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:24.173910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:24.173910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:24.173910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 04:02:24.173910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 04:02:24.173910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 04:02:24.173910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 04:02:24.173910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 04:02:24.173910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 04:02:24.173910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:02:24.173910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:02:24.173910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 619/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 04:02:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:24.174655 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c825ed-493d-4e56-bbd9-67edd37faafc None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:24.275713 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c825ed-493d-4e56-bbd9-67edd37faafc None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:24.304486 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:24.309314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664c02c-c7c2-4ebb-b3e2-17020ed4d253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:24.309978 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664c02c-c7c2-4ebb-b3e2-17020ed4d253 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:24.310263 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664c02c-c7c2-4ebb-b3e2-17020ed4d253 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:24.310576 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664c02c-c7c2-4ebb-b3e2-17020ed4d253 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:24.311324 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664c02c-c7c2-4ebb-b3e2-17020ed4d253 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:24.311632 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664c02c-c7c2-4ebb-b3e2-17020ed4d253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:24.313661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664c02c-c7c2-4ebb-b3e2-17020ed4d253 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:24.321618 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 631/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 04:02:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:24.327446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4691da-79b6-4dc3-87a5-440c103354e2 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:24.403986 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:24.427423 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4691da-79b6-4dc3-87a5-440c103354e2 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:24.460035 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2af2a-e64d-440d-b9a6-e4228636f9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:24.460816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2af2a-e64d-440d-b9a6-e4228636f9fe None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:24.461102 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2af2a-e64d-440d-b9a6-e4228636f9fe None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:24.461382 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd2af2a-e64d-440d-b9a6-e4228636f9fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:24.472467 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2af2a-e64d-440d-b9a6-e4228636f9fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:24.473921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd2af2a-e64d-440d-b9a6-e4228636f9fe None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:24.491934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-bbd2af2a-e64d-440d-b9a6-e4228636f9fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:24.491934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:24.491934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:24.491934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:24.491934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:24.491934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:24.491934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:24.491934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:24.491934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:24.491934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:24.491934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:24.491934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:24.491934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 04:02:24.491934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 04:02:24.491934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 04:02:24.492981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 04:02:24.492981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:24.492981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:24.492981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:24.492981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:24.492981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:24.492981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:24.492981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:24.492981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:24.492981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:24.492981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:24.492981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:24.492981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:24.492981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:24.492981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:24.492981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:24.492981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:24.494131 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 04:02:24.494131 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 04:02:24.494131 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:24.494131 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:02:24.494131 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 620/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:02:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:24.499185 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664c02c-c7c2-4ebb-b3e2-17020ed4d253 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:24.599151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664c02c-c7c2-4ebb-b3e2-17020ed4d253 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:24.632515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7095c7-202b-4acf-841b-7fbed3ada1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:24.633172 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7095c7-202b-4acf-841b-7fbed3ada1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:24.633457 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7095c7-202b-4acf-841b-7fbed3ada1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:24.633734 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7095c7-202b-4acf-841b-7fbed3ada1a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:24.634539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7095c7-202b-4acf-841b-7fbed3ada1a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:24.634854 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7095c7-202b-4acf-841b-7fbed3ada1a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:24.636221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7095c7-202b-4acf-841b-7fbed3ada1a1 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:24.647758 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 632/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 04:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:24.653468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2af2a-e64d-440d-b9a6-e4228636f9fe None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:24.756494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd2af2a-e64d-440d-b9a6-e4228636f9fe None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:24.783202 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:24.791014 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4a5ec-cf2b-4a27-bee7-4d8aeedca1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:24.791835 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4a5ec-cf2b-4a27-bee7-4d8aeedca1b4 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:24.792117 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4a5ec-cf2b-4a27-bee7-4d8aeedca1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:24.792390 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4a5ec-cf2b-4a27-bee7-4d8aeedca1b4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:24.799834 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4a5ec-cf2b-4a27-bee7-4d8aeedca1b4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:24.801387 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4a5ec-cf2b-4a27-bee7-4d8aeedca1b4 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:24.815588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-e0d4a5ec-cf2b-4a27-bee7-4d8aeedca1b4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 04:02:24.815588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:24.815588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:24.815588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:24.815588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:24.815588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:24.815588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:24.815588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:24.815588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:24.815588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:24.815588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:24.815588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:24.815588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 04:02:24.815588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 04:02:24.815588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 04:02:24.815588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 04:02:24.816794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:24.816794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:24.816794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 04:02:24.816794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 04:02:24.816794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 04:02:24.816794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 04:02:24.816794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:24.816794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:24.816794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:24.816794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:24.816794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:24.816794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:24.816794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:24.816794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:24.816794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:24.816794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:24.816794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:24.816794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:24.818158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:24.818158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:24.818158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 04:02:24.818158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 04:02:24.818158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 04:02:24.818158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 04:02:24.818158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 04:02:24.818158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 04:02:24.818158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 04:02:24.818158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:02:24.818158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 621/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:02:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:24.825329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7095c7-202b-4acf-841b-7fbed3ada1a1 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:24.877770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:24.921780 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7095c7-202b-4acf-841b-7fbed3ada1a1 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:24.953441 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1ded3-d8ba-434b-8c00-ed815aa82393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:24.954729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1ded3-d8ba-434b-8c00-ed815aa82393 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:24.955018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1ded3-d8ba-434b-8c00-ed815aa82393 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:24.955298 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d1ded3-d8ba-434b-8c00-ed815aa82393 None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:24.956135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d1ded3-d8ba-434b-8c00-ed815aa82393 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:24.956447 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d1ded3-d8ba-434b-8c00-ed815aa82393 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:24.957864 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d1ded3-d8ba-434b-8c00-ed815aa82393 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:24.975106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 633/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Fri Jun 24 04:02:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:24.981001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4a5ec-cf2b-4a27-bee7-4d8aeedca1b4 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:25.080834 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4a5ec-cf2b-4a27-bee7-4d8aeedca1b4 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:25.114548 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268cea4e-7e94-4d30-8e26-a4b880d9fc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:25.119182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cea4e-7e94-4d30-8e26-a4b880d9fc8f None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:25.119182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cea4e-7e94-4d30-8e26-a4b880d9fc8f None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:25.119182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cea4e-7e94-4d30-8e26-a4b880d9fc8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:25.170535 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:25.182591 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268cea4e-7e94-4d30-8e26-a4b880d9fc8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:25.184046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268cea4e-7e94-4d30-8e26-a4b880d9fc8f None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:25.281026 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 622/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 04:02:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 04:02:25.287169 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1ded3-d8ba-434b-8c00-ed815aa82393 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:25.387400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d1ded3-d8ba-434b-8c00-ed815aa82393 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:25.400744 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:25.421113 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6aa260-3eb1-4c65-a78a-403d2a7038db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:25.421897 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6aa260-3eb1-4c65-a78a-403d2a7038db None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:25.422176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6aa260-3eb1-4c65-a78a-403d2a7038db None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:25.422979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6aa260-3eb1-4c65-a78a-403d2a7038db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:25.432577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6aa260-3eb1-4c65-a78a-403d2a7038db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:25.434011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6aa260-3eb1-4c65-a78a-403d2a7038db None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:25.451729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-1d6aa260-3eb1-4c65-a78a-403d2a7038db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:25.451729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:25.451729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:25.451729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:25.451729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:25.451729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:25.451729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:25.451729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:25.451729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:25.451729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:25.451729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:25.451729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:25.451729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 04:02:25.451729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 04:02:25.451729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 04:02:25.452790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 04:02:25.452790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:25.452790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:25.452790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:25.452790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:25.452790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:25.452790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:25.452790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:25.452790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:25.452790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:25.452790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:25.452790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:25.452790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:25.452790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:25.452790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:25.452790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:25.452790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:25.454240 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 04:02:25.454240 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 04:02:25.454240 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:25.454240 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:25.454240 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 634/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:02:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:25.458964 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268cea4e-7e94-4d30-8e26-a4b880d9fc8f None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:25.545568 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:25.556968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268cea4e-7e94-4d30-8e26-a4b880d9fc8f None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:25.589622 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304f731-2ccc-43b0-8e21-1cba0c8b8a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:25.590297 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304f731-2ccc-43b0-8e21-1cba0c8b8a64 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:25.590606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304f731-2ccc-43b0-8e21-1cba0c8b8a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:25.590883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304f731-2ccc-43b0-8e21-1cba0c8b8a64 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:25.591589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304f731-2ccc-43b0-8e21-1cba0c8b8a64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:25.591893 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304f731-2ccc-43b0-8e21-1cba0c8b8a64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:25.593246 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304f731-2ccc-43b0-8e21-1cba0c8b8a64 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:25.606170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 623/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 04:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:25.612431 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6aa260-3eb1-4c65-a78a-403d2a7038db None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:25.710236 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6aa260-3eb1-4c65-a78a-403d2a7038db None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:25.744480 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a4360-2e5a-43e6-9efa-666f854c1cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:25.745273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a4360-2e5a-43e6-9efa-666f854c1cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:25.745563 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a4360-2e5a-43e6-9efa-666f854c1cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:25.746106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a4360-2e5a-43e6-9efa-666f854c1cd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:25.752740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a4360-2e5a-43e6-9efa-666f854c1cd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:25.754185 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a4360-2e5a-43e6-9efa-666f854c1cd0 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:25.768239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-1c2a4360-2e5a-43e6-9efa-666f854c1cd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 04:02:25.768239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:25.768239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:25.768239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:25.768239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:25.768239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:25.768239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:25.768239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:25.768239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:25.768239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:25.768239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:25.768239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:25.768239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 04:02:25.768239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 04:02:25.768239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 04:02:25.769176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 04:02:25.769176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:25.769176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:25.769176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:25.769176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:25.769176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:25.769176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:25.769176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:25.769176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:25.769176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:25.769176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:25.769176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:25.769176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:25.769176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:25.769176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:25.769176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:25.769176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:25.770114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 04:02:25.770114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 04:02:25.770114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 04:02:25.770114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 04:02:25.770114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 04:02:25.770114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 04:02:25.770114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 04:02:25.770114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:25.770114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 635/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 04:02:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:25.775341 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304f731-2ccc-43b0-8e21-1cba0c8b8a64 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:25.873777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304f731-2ccc-43b0-8e21-1cba0c8b8a64 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:25.907549 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f797a24-133a-4479-afaf-b1303261d0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:25.908239 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f797a24-133a-4479-afaf-b1303261d0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:25.908480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f797a24-133a-4479-afaf-b1303261d0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:25.908760 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f797a24-133a-4479-afaf-b1303261d0d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:25.909544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f797a24-133a-4479-afaf-b1303261d0d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:25.909854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f797a24-133a-4479-afaf-b1303261d0d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:25.911313 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f797a24-133a-4479-afaf-b1303261d0d1 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:25.924514 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 624/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 04:02:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:25.930604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a4360-2e5a-43e6-9efa-666f854c1cd0 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:25.957162 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:26.028498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a4360-2e5a-43e6-9efa-666f854c1cd0 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:26.061426 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2527af1-9bd7-40fd-b45c-67ae44de7d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:26.062079 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2527af1-9bd7-40fd-b45c-67ae44de7d5a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:26.062462 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2527af1-9bd7-40fd-b45c-67ae44de7d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:26.062677 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2527af1-9bd7-40fd-b45c-67ae44de7d5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:26.064239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2527af1-9bd7-40fd-b45c-67ae44de7d5a None admin] RBAC: Adding query filter params (role.id=f0f7e28cbe314921afdfb657b163436f, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=7662e61619d44c92af3ffacbacba3913) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:26.064793 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2527af1-9bd7-40fd-b45c-67ae44de7d5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:26.066897 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2527af1-9bd7-40fd-b45c-67ae44de7d5a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:26.072356 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 636/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 04:02:25 2022] GET /identity/v3/role_assignments?user.id=7662e61619d44c92af3ffacbacba3913&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=f0f7e28cbe314921afdfb657b163436f => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:28.096698 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc94208-c2a7-4dff-8739-6c1d560309ae None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:28.097265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc94208-c2a7-4dff-8739-6c1d560309ae None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:28.097379 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc94208-c2a7-4dff-8739-6c1d560309ae None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:28.098728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 625/1261] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:02:28.104150 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3ee62-1b7a-42a4-9d90-127f2fd5be94 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:28.104560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3ee62-1b7a-42a4-9d90-127f2fd5be94 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:28.104769 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3ee62-1b7a-42a4-9d90-127f2fd5be94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:28.121820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1db3ee62-1b7a-42a4-9d90-127f2fd5be94 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:02:28.123033 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1db3ee62-1b7a-42a4-9d90-127f2fd5be94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:02:28.191220 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 637/1262] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:02:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:02:28.213688 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc94208-c2a7-4dff-8739-6c1d560309ae None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:28.225206 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:28.316157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc94208-c2a7-4dff-8739-6c1d560309ae None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:28.351554 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26b5db-af15-447e-a79a-3ca134199cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:28.352551 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26b5db-af15-447e-a79a-3ca134199cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:28.352868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26b5db-af15-447e-a79a-3ca134199cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:28.353141 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26b5db-af15-447e-a79a-3ca134199cd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:28.365350 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a26b5db-af15-447e-a79a-3ca134199cd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 04:02:28.373286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-2a26b5db-af15-447e-a79a-3ca134199cd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:02:28.373286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:28.373286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:28.373286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:28.373286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:28.373286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:28.373286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:28.373286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:28.373286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:28.373286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:28.373286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:28.373286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:28.373286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 04:02:28.373286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 04:02:28.373286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 04:02:28.373286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 04:02:28.381759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 04:02:28.381759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 04:02:28.381759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 04:02:28.381759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 04:02:28.381759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:28.381759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:28.381759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:28.381759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:28.381759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:28.381759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:28.381759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:28.381759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:28.381759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:28.381759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:28.381759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:28.381759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:28.382854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:28.382854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:28.382854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:28.382854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:28.382854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 04:02:28.382854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 04:02:28.382854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 04:02:28.382854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 04:02:28.382854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 04:02:28.382854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 04:02:28.382854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:02:28.382854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:02:28.382854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 626/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 04:02:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:28.383792 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3ee62-1b7a-42a4-9d90-127f2fd5be94 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:28.399390 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:28.480578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3ee62-1b7a-42a4-9d90-127f2fd5be94 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:28.514205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a50c0-b19c-43e5-b45e-eb4e80d34629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:28.514899 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a50c0-b19c-43e5-b45e-eb4e80d34629 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:28.515189 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a50c0-b19c-43e5-b45e-eb4e80d34629 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:28.515472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a50c0-b19c-43e5-b45e-eb4e80d34629 None admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:28.516735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a50c0-b19c-43e5-b45e-eb4e80d34629 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:28.517047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a50c0-b19c-43e5-b45e-eb4e80d34629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:28.518463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a50c0-b19c-43e5-b45e-eb4e80d34629 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:28.526336 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 638/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 04:02:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:28.532727 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26b5db-af15-447e-a79a-3ca134199cd9 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:28.598702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:28.630490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26b5db-af15-447e-a79a-3ca134199cd9 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:28.663689 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf5503-6cd7-4986-ac03-032f5a8ae221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:28.664473 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf5503-6cd7-4986-ac03-032f5a8ae221 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:28.664760 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf5503-6cd7-4986-ac03-032f5a8ae221 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:28.665071 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf5503-6cd7-4986-ac03-032f5a8ae221 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:28.675544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf5503-6cd7-4986-ac03-032f5a8ae221 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:28.687474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf5503-6cd7-4986-ac03-032f5a8ae221 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:28.718776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-afaf5503-6cd7-4986-ac03-032f5a8ae221 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:28.718776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:28.718776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:28.718776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:28.718776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:28.718776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:28.718776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:28.718776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:28.718776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:28.718776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:28.718776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:28.718776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:28.718776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 04:02:28.718776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 04:02:28.718776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 04:02:28.720777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 04:02:28.720777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:28.720777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:28.720777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:28.720777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:28.720777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:28.720777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:28.720777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:28.720777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:28.720777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:28.720777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:28.720777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:28.720777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:28.720777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:28.720777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:28.720777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:28.720777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:28.721699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 04:02:28.721699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 04:02:28.721699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:28.721699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:02:28.721699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 627/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:02:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 04:02:28.727670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a50c0-b19c-43e5-b45e-eb4e80d34629 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:28.811101 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:28.830753 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a50c0-b19c-43e5-b45e-eb4e80d34629 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:28.864657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a54ae1-77e4-4b64-b44c-0f8ba4c42c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:28.865251 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a54ae1-77e4-4b64-b44c-0f8ba4c42c37 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:28.865535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a54ae1-77e4-4b64-b44c-0f8ba4c42c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:28.865808 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a54ae1-77e4-4b64-b44c-0f8ba4c42c37 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:28.866559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a54ae1-77e4-4b64-b44c-0f8ba4c42c37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:28.866865 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a54ae1-77e4-4b64-b44c-0f8ba4c42c37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:28.868228 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a54ae1-77e4-4b64-b44c-0f8ba4c42c37 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:28.880310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 639/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 04:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:28.886005 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf5503-6cd7-4986-ac03-032f5a8ae221 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:28.985557 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf5503-6cd7-4986-ac03-032f5a8ae221 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:29.005397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:29.020655 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b48d96-ffe5-413c-9f32-58709f594217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:29.021461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b48d96-ffe5-413c-9f32-58709f594217 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:29.021739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b48d96-ffe5-413c-9f32-58709f594217 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:29.022018 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b48d96-ffe5-413c-9f32-58709f594217 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:29.030235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b48d96-ffe5-413c-9f32-58709f594217 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:29.031874 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b48d96-ffe5-413c-9f32-58709f594217 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:29.047023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-60b48d96-ffe5-413c-9f32-58709f594217 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 04:02:29.047023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:29.047023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:29.047023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:29.047023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:29.047023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:29.047023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:29.047023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:29.047023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:29.047023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:29.047023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:29.047023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:29.047023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 04:02:29.047023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 04:02:29.047023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 04:02:29.047023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 04:02:29.048226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:29.048226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:29.048226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 04:02:29.048226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 04:02:29.048226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 04:02:29.048226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 04:02:29.048226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:29.048226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:29.048226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:29.048226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:29.048226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:29.048226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:29.048226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:29.048226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:29.048226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:29.048226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:29.048226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:29.048226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:29.049404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:29.049404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:29.049404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 04:02:29.049404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 04:02:29.049404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 04:02:29.049404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 04:02:29.049404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 04:02:29.049404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 04:02:29.049404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 04:02:29.049404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:02:29.049404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 628/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:02:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:29.054476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a54ae1-77e4-4b64-b44c-0f8ba4c42c37 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:29.155077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a54ae1-77e4-4b64-b44c-0f8ba4c42c37 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:29.188483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0df65a-1507-4719-b1f4-fbfe2987e09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:29.189156 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0df65a-1507-4719-b1f4-fbfe2987e09d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:29.189440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0df65a-1507-4719-b1f4-fbfe2987e09d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:29.189717 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0df65a-1507-4719-b1f4-fbfe2987e09d None admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:29.190589 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0df65a-1507-4719-b1f4-fbfe2987e09d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:29.190900 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0df65a-1507-4719-b1f4-fbfe2987e09d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:29.192326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0df65a-1507-4719-b1f4-fbfe2987e09d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:29.201198 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:29.209570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 640/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Fri Jun 24 04:02:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:29.215277 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b48d96-ffe5-413c-9f32-58709f594217 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:29.314656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b48d96-ffe5-413c-9f32-58709f594217 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:29.347580 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a18242-277f-416d-b381-029aace4eaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:29.348380 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a18242-277f-416d-b381-029aace4eaa3 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:29.348707 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a18242-277f-416d-b381-029aace4eaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:29.349025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a18242-277f-416d-b381-029aace4eaa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:29.360436 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a18242-277f-416d-b381-029aace4eaa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:29.361899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a18242-277f-416d-b381-029aace4eaa3 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:29.379880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-04a18242-277f-416d-b381-029aace4eaa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:29.379880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:29.379880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:29.379880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:29.379880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:29.379880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:29.379880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:29.379880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:29.379880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:29.379880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:29.379880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:29.379880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:29.379880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 04:02:29.379880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 04:02:29.379880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 04:02:29.380959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 04:02:29.380959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:29.380959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:29.380959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:29.380959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:29.380959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:29.380959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:29.380959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:29.380959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:29.380959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:29.380959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:29.380959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:29.380959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:29.380959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:29.380959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:29.380959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:29.380959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:29.381867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 04:02:29.381867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 04:02:29.381867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:29.381867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:02:29.381867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 629/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:29.386774 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0df65a-1507-4719-b1f4-fbfe2987e09d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:29.485857 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0df65a-1507-4719-b1f4-fbfe2987e09d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:29.519801 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be15f62-d769-4123-a8c5-54f275cb10b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:29.520455 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be15f62-d769-4123-a8c5-54f275cb10b8 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:29.520740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be15f62-d769-4123-a8c5-54f275cb10b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:29.521018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be15f62-d769-4123-a8c5-54f275cb10b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:29.521737 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be15f62-d769-4123-a8c5-54f275cb10b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:29.522047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be15f62-d769-4123-a8c5-54f275cb10b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:29.523451 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be15f62-d769-4123-a8c5-54f275cb10b8 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:29.534528 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 641/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 04:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:29.540085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a18242-277f-416d-b381-029aace4eaa3 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:29.572974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:29.639398 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a18242-277f-416d-b381-029aace4eaa3 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:29.674422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c67d1-8f63-4cfd-bcdf-a9c4cb642299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:29.675194 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c67d1-8f63-4cfd-bcdf-a9c4cb642299 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:29.675472 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c67d1-8f63-4cfd-bcdf-a9c4cb642299 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:29.675741 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c67d1-8f63-4cfd-bcdf-a9c4cb642299 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:29.682150 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c67d1-8f63-4cfd-bcdf-a9c4cb642299 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:29.683627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c67d1-8f63-4cfd-bcdf-a9c4cb642299 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:29.697734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-429c67d1-8f63-4cfd-bcdf-a9c4cb642299 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 04:02:29.697734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:29.697734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:29.697734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:29.697734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:29.697734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:29.697734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:29.697734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:29.697734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:29.697734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:29.697734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:29.697734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:29.697734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 04:02:29.697734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 04:02:29.697734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 04:02:29.704120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 04:02:29.704120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:29.704120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:29.704120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:29.704120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:29.704120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:29.704120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:29.704120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:29.704120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:29.704120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:29.704120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:29.704120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:29.704120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:29.704120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:29.704120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:29.704120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:29.704120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:29.705339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 04:02:29.705339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 04:02:29.705339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 04:02:29.705339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 04:02:29.705339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 04:02:29.705339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 04:02:29.705339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 04:02:29.705339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:02:29.705339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 630/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 04:02:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:29.707211 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be15f62-d769-4123-a8c5-54f275cb10b8 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:29.765316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:29.806461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be15f62-d769-4123-a8c5-54f275cb10b8 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:29.838963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4600f3-56be-4774-9162-a007a3e3eb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:29.839618 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4600f3-56be-4774-9162-a007a3e3eb99 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:29.839954 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4600f3-56be-4774-9162-a007a3e3eb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:29.840228 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4600f3-56be-4774-9162-a007a3e3eb99 None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:29.841008 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4600f3-56be-4774-9162-a007a3e3eb99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:29.841322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4600f3-56be-4774-9162-a007a3e3eb99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:29.842782 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4600f3-56be-4774-9162-a007a3e3eb99 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:29.855202 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 642/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 04:02:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:29.861797 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c67d1-8f63-4cfd-bcdf-a9c4cb642299 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:29.952348 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:29.961043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c67d1-8f63-4cfd-bcdf-a9c4cb642299 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:29.995528 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62376212-f1bc-401c-baca-b36ce6908e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:29.996667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62376212-f1bc-401c-baca-b36ce6908e7c None admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:29.996952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62376212-f1bc-401c-baca-b36ce6908e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:29.997233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62376212-f1bc-401c-baca-b36ce6908e7c None admin] PATH_INFO: `/v3/projects/68d376986e854a4bb334676572340b4c/users/7662e61619d44c92af3ffacbacba3913/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:29.999216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62376212-f1bc-401c-baca-b36ce6908e7c None admin] RBAC: Authorizing `identity:create_grant(project_id=68d376986e854a4bb334676572340b4c, user_id=7662e61619d44c92af3ffacbacba3913, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:30.000829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62376212-f1bc-401c-baca-b36ce6908e7c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:30.118250 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 631/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Jun 24 04:02:29 2022] PUT /identity/v3/projects/68d376986e854a4bb334676572340b4c/users/7662e61619d44c92af3ffacbacba3913/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:02:32.201634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa9253-f0a8-4028-a98a-1ad845933b82 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:32.202225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa9253-f0a8-4028-a98a-1ad845933b82 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:32.202225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa9253-f0a8-4028-a98a-1ad845933b82 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:32.204239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 643/1274] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:02:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:02:32.209597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410b3cc-3586-4ba5-83bf-67cfa68e556c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:32.210004 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410b3cc-3586-4ba5-83bf-67cfa68e556c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:32.210151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410b3cc-3586-4ba5-83bf-67cfa68e556c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:32.226277 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f410b3cc-3586-4ba5-83bf-67cfa68e556c None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:02:32.227537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f410b3cc-3586-4ba5-83bf-67cfa68e556c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:02:32.295480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 632/1275] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:02:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:02:32.320032 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa9253-f0a8-4028-a98a-1ad845933b82 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:32.421208 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa9253-f0a8-4028-a98a-1ad845933b82 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:32.439795 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:32.456430 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e014680-a4c6-4d3c-a497-8655a951a0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:32.457220 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e014680-a4c6-4d3c-a497-8655a951a0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:32.457504 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e014680-a4c6-4d3c-a497-8655a951a0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:32.457780 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e014680-a4c6-4d3c-a497-8655a951a0e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:32.467626 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e014680-a4c6-4d3c-a497-8655a951a0e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 04:02:32.475440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-4e014680-a4c6-4d3c-a497-8655a951a0e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:02:32.475440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:32.475440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:32.475440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:32.475440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:32.475440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:32.475440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:32.475440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:32.475440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:32.475440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:32.475440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:32.475440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:32.475440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 04:02:32.475440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 04:02:32.475440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 04:02:32.475440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 04:02:32.476643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 04:02:32.476643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 04:02:32.476643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 04:02:32.476643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 04:02:32.476643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:32.476643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:32.476643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:32.476643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:32.476643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:32.476643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:32.476643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:32.476643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:32.476643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:32.476643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:32.476643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:32.476643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:32.477832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:32.477832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:32.477832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:32.477832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:32.477832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 04:02:32.477832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 04:02:32.477832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 04:02:32.477832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 04:02:32.477832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 04:02:32.477832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 04:02:32.477832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 04:02:32.477832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:32.477832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 644/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Jun 24 04:02:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:32.482933 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410b3cc-3586-4ba5-83bf-67cfa68e556c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:32.562551 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:32.581368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410b3cc-3586-4ba5-83bf-67cfa68e556c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:32.615068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbc200-bcef-4c8f-bb1e-eeeb1b2e0842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:32.615730 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbc200-bcef-4c8f-bb1e-eeeb1b2e0842 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:32.616015 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbc200-bcef-4c8f-bb1e-eeeb1b2e0842 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:32.616292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbc200-bcef-4c8f-bb1e-eeeb1b2e0842 None admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:32.617030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fbc200-bcef-4c8f-bb1e-eeeb1b2e0842 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:32.617337 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fbc200-bcef-4c8f-bb1e-eeeb1b2e0842 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:32.618842 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fbc200-bcef-4c8f-bb1e-eeeb1b2e0842 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:32.627703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 633/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Jun 24 04:02:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:32.633614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e014680-a4c6-4d3c-a497-8655a951a0e5 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:32.732935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e014680-a4c6-4d3c-a497-8655a951a0e5 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:32.766576 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced4532-d4c7-438b-bc6a-f5f089603051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:32.767370 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced4532-d4c7-438b-bc6a-f5f089603051 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:32.767655 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced4532-d4c7-438b-bc6a-f5f089603051 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:32.767932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced4532-d4c7-438b-bc6a-f5f089603051 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:32.778168 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced4532-d4c7-438b-bc6a-f5f089603051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:32.779699 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ced4532-d4c7-438b-bc6a-f5f089603051 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:32.797729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-0ced4532-d4c7-438b-bc6a-f5f089603051 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:32.797729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:32.797729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:32.797729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:32.797729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:32.797729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:32.797729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:32.797729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:32.797729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:32.797729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:32.797729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:32.797729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:32.797729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 04:02:32.797729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 04:02:32.797729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 04:02:32.798837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 04:02:32.798837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:32.798837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:32.798837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:32.798837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:32.798837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:32.798837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:32.798837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:32.798837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:32.798837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:32.798837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:32.798837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:32.798837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:32.798837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:32.798837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:32.798837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:32.798837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:32.799855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 04:02:32.799855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 04:02:32.799855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:32.799855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:32.799855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 645/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:02:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:32.804931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbc200-bcef-4c8f-bb1e-eeeb1b2e0842 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:32.904197 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbc200-bcef-4c8f-bb1e-eeeb1b2e0842 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:32.938117 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fc9c8-082f-449f-bc91-eb5c0d23f384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:32.938802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fc9c8-082f-449f-bc91-eb5c0d23f384 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:32.939116 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fc9c8-082f-449f-bc91-eb5c0d23f384 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:32.939397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fc9c8-082f-449f-bc91-eb5c0d23f384 None admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:32.940110 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fc9c8-082f-449f-bc91-eb5c0d23f384 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:32.940418 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fc9c8-082f-449f-bc91-eb5c0d23f384 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:32.941769 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fc9c8-082f-449f-bc91-eb5c0d23f384 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:32.953220 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 634/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 04:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:32.959293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced4532-d4c7-438b-bc6a-f5f089603051 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:32.990284 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:33.059961 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced4532-d4c7-438b-bc6a-f5f089603051 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:33.093957 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb92c7-72fd-4cda-aa07-cf7be3c1f71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:33.095306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb92c7-72fd-4cda-aa07-cf7be3c1f71d None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:33.095592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb92c7-72fd-4cda-aa07-cf7be3c1f71d None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:33.095865 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb92c7-72fd-4cda-aa07-cf7be3c1f71d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:33.102717 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb92c7-72fd-4cda-aa07-cf7be3c1f71d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:33.104156 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb92c7-72fd-4cda-aa07-cf7be3c1f71d None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:33.118991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-38eb92c7-72fd-4cda-aa07-cf7be3c1f71d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 04:02:33.118991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:33.118991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:33.118991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:33.118991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:33.118991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:33.118991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:33.118991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:33.118991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:33.118991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:33.118991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:33.118991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:33.118991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 04:02:33.118991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 04:02:33.118991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 04:02:33.118991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 04:02:33.120135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:33.120135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:33.120135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 04:02:33.120135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 04:02:33.120135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 04:02:33.120135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 04:02:33.120135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:33.120135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:33.120135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:33.120135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:33.120135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:33.120135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:33.120135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:33.120135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:33.120135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:33.120135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:33.120135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:33.120135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:33.121368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:33.121368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:33.121368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 04:02:33.121368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 04:02:33.121368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 04:02:33.121368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 04:02:33.121368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 04:02:33.121368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 04:02:33.121368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 04:02:33.121368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:33.121368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 646/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:02:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:33.126352 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fc9c8-082f-449f-bc91-eb5c0d23f384 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:33.130993 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:33.227096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fc9c8-082f-449f-bc91-eb5c0d23f384 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:33.260270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034522f-43e7-4182-baff-99e04f8e9ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:33.260928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034522f-43e7-4182-baff-99e04f8e9ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:33.261208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034522f-43e7-4182-baff-99e04f8e9ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:33.261482 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034522f-43e7-4182-baff-99e04f8e9ad8 None admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:33.262307 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034522f-43e7-4182-baff-99e04f8e9ad8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:33.262640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034522f-43e7-4182-baff-99e04f8e9ad8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:33.264076 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034522f-43e7-4182-baff-99e04f8e9ad8 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:33.283262 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 635/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Fri Jun 24 04:02:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:33.289010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb92c7-72fd-4cda-aa07-cf7be3c1f71d None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:33.380261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:33.389590 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb92c7-72fd-4cda-aa07-cf7be3c1f71d None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:33.422731 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2de9d-eb99-4418-bc16-b1560e44d558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:33.423371 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2de9d-eb99-4418-bc16-b1560e44d558 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:33.423657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2de9d-eb99-4418-bc16-b1560e44d558 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:33.423935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2de9d-eb99-4418-bc16-b1560e44d558 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:33.489475 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2de9d-eb99-4418-bc16-b1560e44d558 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:33.490890 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2de9d-eb99-4418-bc16-b1560e44d558 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:33.579107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:33.590065 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 647/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Jun 24 04:02:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 04:02:33.599000 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034522f-43e7-4182-baff-99e04f8e9ad8 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:33.659695 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:33.697398 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034522f-43e7-4182-baff-99e04f8e9ad8 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:33.731845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449aeb9-a758-46b8-8c99-5ecee1daf25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:33.732626 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449aeb9-a758-46b8-8c99-5ecee1daf25c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:33.732910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449aeb9-a758-46b8-8c99-5ecee1daf25c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:33.733184 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449aeb9-a758-46b8-8c99-5ecee1daf25c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:33.744542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449aeb9-a758-46b8-8c99-5ecee1daf25c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:33.746213 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449aeb9-a758-46b8-8c99-5ecee1daf25c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:33.764193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-4449aeb9-a758-46b8-8c99-5ecee1daf25c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:33.764193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:33.764193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:33.764193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:33.764193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:33.764193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:33.764193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:33.764193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:33.764193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:33.764193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:33.764193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:33.764193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:33.764193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 04:02:33.764193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 04:02:33.764193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 04:02:33.767563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 04:02:33.767563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:33.767563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:33.767563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:33.767563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:33.767563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:33.767563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:33.767563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:33.767563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:33.767563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:33.767563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:33.767563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:33.767563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:33.767563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:33.767563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:33.767563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:33.767563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:33.768660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 04:02:33.768660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 04:02:33.768660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 04:02:33.768660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:02:33.768660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 636/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:02:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:33.771150 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2de9d-eb99-4418-bc16-b1560e44d558 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:33.868164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2de9d-eb99-4418-bc16-b1560e44d558 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:33.903203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e6b7c-ff9d-4965-818e-2432f0d7898f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:33.903911 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e6b7c-ff9d-4965-818e-2432f0d7898f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:33.904237 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e6b7c-ff9d-4965-818e-2432f0d7898f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:33.904608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e6b7c-ff9d-4965-818e-2432f0d7898f None admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:33.905415 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e6b7c-ff9d-4965-818e-2432f0d7898f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:33.905775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e6b7c-ff9d-4965-818e-2432f0d7898f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:33.907269 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e6b7c-ff9d-4965-818e-2432f0d7898f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:33.918337 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 648/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Jun 24 04:02:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:33.924989 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449aeb9-a758-46b8-8c99-5ecee1daf25c None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:34.025150 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449aeb9-a758-46b8-8c99-5ecee1daf25c None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:34.039202 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:34.059295 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221816e3-b3ef-4ab5-b4e1-219e8489aa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:34.060082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221816e3-b3ef-4ab5-b4e1-219e8489aa8a None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:34.060369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221816e3-b3ef-4ab5-b4e1-219e8489aa8a None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:34.060643 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221816e3-b3ef-4ab5-b4e1-219e8489aa8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:34.067820 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221816e3-b3ef-4ab5-b4e1-219e8489aa8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:34.070337 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221816e3-b3ef-4ab5-b4e1-219e8489aa8a None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:34.084214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application [None req-221816e3-b3ef-4ab5-b4e1-219e8489aa8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 04:02:34.084214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:34.084214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:34.084214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:34.084214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:34.084214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:34.084214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:34.084214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:34.084214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:34.084214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:34.084214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:34.084214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:34.084214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 04:02:34.084214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 04:02:34.084214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 04:02:34.085367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 04:02:34.085367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:34.085367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:34.085367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:34.085367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:34.085367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:34.085367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:34.085367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:34.085367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:34.085367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:34.085367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:34.085367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:34.085367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:34.085367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:34.085367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:34.085367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:34.085367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:34.086485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 04:02:34.086485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 04:02:34.086485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 04:02:34.086485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 04:02:34.086485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 04:02:34.086485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 04:02:34.086485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 04:02:34.086485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: ERROR keystone.server.flask.application Jun 24 04:02:34.086485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 637/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Jun 24 04:02:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:34.091200 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e6b7c-ff9d-4965-818e-2432f0d7898f None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:34.142516 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:34.192290 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e6b7c-ff9d-4965-818e-2432f0d7898f None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:34.226743 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5a4b4-512b-48ea-a3ac-5620add91c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:34.227396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5a4b4-512b-48ea-a3ac-5620add91c5a None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:34.227681 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5a4b4-512b-48ea-a3ac-5620add91c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:34.227953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5a4b4-512b-48ea-a3ac-5620add91c5a None admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:34.228738 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5a4b4-512b-48ea-a3ac-5620add91c5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:34.229041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5a4b4-512b-48ea-a3ac-5620add91c5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:34.230485 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5a4b4-512b-48ea-a3ac-5620add91c5a None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:34.243229 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 649/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Jun 24 04:02:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:34.249715 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221816e3-b3ef-4ab5-b4e1-219e8489aa8a None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:34.348643 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221816e3-b3ef-4ab5-b4e1-219e8489aa8a None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:34.382810 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8ee7c-8cab-4e16-bdfc-19c4d2423698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:34.383445 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8ee7c-8cab-4e16-bdfc-19c4d2423698 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:34.383749 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8ee7c-8cab-4e16-bdfc-19c4d2423698 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:34.384022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8ee7c-8cab-4e16-bdfc-19c4d2423698 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:34.384978 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8ee7c-8cab-4e16-bdfc-19c4d2423698 None admin] RBAC: Adding query filter params (role.id=f0f7e28cbe314921afdfb657b163436f, scope.project.id=68d376986e854a4bb334676572340b4c, user.id=7662e61619d44c92af3ffacbacba3913) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:34.385296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8ee7c-8cab-4e16-bdfc-19c4d2423698 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:34.386733 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8ee7c-8cab-4e16-bdfc-19c4d2423698 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:34.392625 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 638/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Jun 24 04:02:34 2022] GET /identity/v3/role_assignments?user.id=7662e61619d44c92af3ffacbacba3913&scope.project.id=68d376986e854a4bb334676572340b4c&role.id=f0f7e28cbe314921afdfb657b163436f => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:36.480405 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a6ab6-daeb-4dcb-87ca-b034120efd3f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:36.480958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a6ab6-daeb-4dcb-87ca-b034120efd3f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:36.480958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a6ab6-daeb-4dcb-87ca-b034120efd3f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:36.482344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 650/1288] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 04:02:36.487987 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be6e70-8873-44d1-83b4-a4b9c4ee6b29 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:36.488430 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be6e70-8873-44d1-83b4-a4b9c4ee6b29 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:36.488557 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be6e70-8873-44d1-83b4-a4b9c4ee6b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:36.505159 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-54be6e70-8873-44d1-83b4-a4b9c4ee6b29 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:02:36.506335 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-54be6e70-8873-44d1-83b4-a4b9c4ee6b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:02:36.535922 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:36.583066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 639/1289] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:02:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:02:36.606809 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a6ab6-daeb-4dcb-87ca-b034120efd3f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:36.705739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a6ab6-daeb-4dcb-87ca-b034120efd3f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:36.716107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:36.739860 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e5c83-71b5-44ef-ac33-330b1dd58d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:36.740593 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e5c83-71b5-44ef-ac33-330b1dd58d69 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:36.740877 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e5c83-71b5-44ef-ac33-330b1dd58d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:36.741153 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e5c83-71b5-44ef-ac33-330b1dd58d69 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:36.741797 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e5c83-71b5-44ef-ac33-330b1dd58d69 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:36.743187 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e5c83-71b5-44ef-ac33-330b1dd58d69 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:36.755334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-fb3e5c83-71b5-44ef-ac33-330b1dd58d69 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 04:02:36.755334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:36.755334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 04:02:36.755334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 04:02:36.755334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 04:02:36.755334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 04:02:36.755334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 04:02:36.755334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 04:02:36.755334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 04:02:36.755334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:36.755334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 04:02:36.755334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:36.755334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:36.755334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:36.755334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:36.755334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:36.755334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:36.756552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:36.756552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:36.756552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:36.756552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:36.756552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:36.756552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:36.756552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 04:02:36.756552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 04:02:36.756552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 04:02:36.756552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 04:02:36.756552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:36.756552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:36.756552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:36.756552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:36.756552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:36.756552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:36.757573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:36.757573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:36.757573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:36.757573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:36.757573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:36.757573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:36.757573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:36.757573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:36.757573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:36.757573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:36.757573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 04:02:36.757573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 04:02:36.757573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 04:02:36.757573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:36.757573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 651/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jun 24 04:02:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:36.762884 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be6e70-8873-44d1-83b4-a4b9c4ee6b29 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:36.861099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be6e70-8873-44d1-83b4-a4b9c4ee6b29 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:36.894895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02876a72-2643-424e-b727-fbb749f407f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:36.895556 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02876a72-2643-424e-b727-fbb749f407f9 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:36.895845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02876a72-2643-424e-b727-fbb749f407f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:36.896129 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02876a72-2643-424e-b727-fbb749f407f9 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:36.896931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02876a72-2643-424e-b727-fbb749f407f9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:36.897247 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02876a72-2643-424e-b727-fbb749f407f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:36.898643 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02876a72-2643-424e-b727-fbb749f407f9 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:36.903389 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 640/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Fri Jun 24 04:02:36 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:36.909836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e5c83-71b5-44ef-ac33-330b1dd58d69 None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:37.008674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e5c83-71b5-44ef-ac33-330b1dd58d69 None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:37.042770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535a1623-83d9-45db-a740-9af5a0da7041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:37.043532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a1623-83d9-45db-a740-9af5a0da7041 None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:37.043816 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a1623-83d9-45db-a740-9af5a0da7041 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:37.044094 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535a1623-83d9-45db-a740-9af5a0da7041 None admin] PATH_INFO: `/v3/services` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:37.044805 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a1623-83d9-45db-a740-9af5a0da7041 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:37.045114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a1623-83d9-45db-a740-9af5a0da7041 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:37.046515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535a1623-83d9-45db-a740-9af5a0da7041 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:37.051770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 652/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Fri Jun 24 04:02:36 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:39.125926 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e97bc9-3739-4e52-b733-c24f2307c6b7 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:39.126482 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e97bc9-3739-4e52-b733-c24f2307c6b7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:39.126482 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e97bc9-3739-4e52-b733-c24f2307c6b7 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:39.127883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 641/1293] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:02:39.133029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0d737-092a-43b9-bdb0-97054c8e7df5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:39.133530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0d737-092a-43b9-bdb0-97054c8e7df5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:39.133686 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0d737-092a-43b9-bdb0-97054c8e7df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:39.151265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1ad0d737-092a-43b9-bdb0-97054c8e7df5 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:02:39.152459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1ad0d737-092a-43b9-bdb0-97054c8e7df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:02:39.181916 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:39.218271 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 653/1294] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:02:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:02:39.242363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e97bc9-3739-4e52-b733-c24f2307c6b7 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:39.273813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:39.343653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e97bc9-3739-4e52-b733-c24f2307c6b7 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:39.377975 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea141198-4e7b-4609-9130-0c2bafd422bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:39.378657 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea141198-4e7b-4609-9130-0c2bafd422bc None admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:39.378946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea141198-4e7b-4609-9130-0c2bafd422bc None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:39.379226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea141198-4e7b-4609-9130-0c2bafd422bc None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:39.379946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea141198-4e7b-4609-9130-0c2bafd422bc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:39.381314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea141198-4e7b-4609-9130-0c2bafd422bc None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:39.481347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 642/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Jun 24 04:02:39 2022] POST /identity/v3/services => generated 246 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:02:41.517507 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f2d95-9a20-4901-9600-7995cde7f55b None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:41.518072 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f2d95-9a20-4901-9600-7995cde7f55b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:41.518072 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f2d95-9a20-4901-9600-7995cde7f55b None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:41.519475 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 654/1296] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:02:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:02:41.524566 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edc639-9edc-4e9c-98a4-49d52c84cc4b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:41.524968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edc639-9edc-4e9c-98a4-49d52c84cc4b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:41.525122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edc639-9edc-4e9c-98a4-49d52c84cc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:41.541882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-98edc639-9edc-4e9c-98a4-49d52c84cc4b None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:02:41.543098 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-98edc639-9edc-4e9c-98a4-49d52c84cc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:02:41.614534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 643/1297] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:02:41 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:02:41.638867 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f2d95-9a20-4901-9600-7995cde7f55b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:41.739227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f2d95-9a20-4901-9600-7995cde7f55b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:41.774778 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f053862-db5c-4b83-bc9e-64e33891f03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:41.775635 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f053862-db5c-4b83-bc9e-64e33891f03b None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:41.776017 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f053862-db5c-4b83-bc9e-64e33891f03b None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:41.776393 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f053862-db5c-4b83-bc9e-64e33891f03b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:41.777217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f053862-db5c-4b83-bc9e-64e33891f03b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:41.778767 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f053862-db5c-4b83-bc9e-64e33891f03b None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:41.791055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-6f053862-db5c-4b83-bc9e-64e33891f03b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 04:02:41.791055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:41.791055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 04:02:41.791055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 04:02:41.791055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 04:02:41.791055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 04:02:41.791055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 04:02:41.791055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 04:02:41.791055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 04:02:41.791055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:41.791055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 04:02:41.791055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:41.791055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:41.791055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:41.791055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:41.791055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:41.791055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:41.792379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:41.792379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:41.792379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:41.792379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:41.792379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:41.792379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:41.792379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 04:02:41.792379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 04:02:41.792379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 04:02:41.792379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 04:02:41.792379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:41.792379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:41.792379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:41.792379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:41.792379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:41.792379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:41.793429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:41.793429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:41.793429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:41.793429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:41.793429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:41.793429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:41.793429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:41.793429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:41.793429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:41.793429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:41.793429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 04:02:41.793429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 04:02:41.793429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 04:02:41.793429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:41.793429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 655/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jun 24 04:02:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:41.798817 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edc639-9edc-4e9c-98a4-49d52c84cc4b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:41.817515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:41.896826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edc639-9edc-4e9c-98a4-49d52c84cc4b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:41.929503 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa689c1-8b38-4f6e-822b-28a34a8fce22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:41.930161 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa689c1-8b38-4f6e-822b-28a34a8fce22 None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:41.930473 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa689c1-8b38-4f6e-822b-28a34a8fce22 None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:41.930793 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa689c1-8b38-4f6e-822b-28a34a8fce22 None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:41.931505 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa689c1-8b38-4f6e-822b-28a34a8fce22 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:41.932020 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa689c1-8b38-4f6e-822b-28a34a8fce22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:41.933784 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa689c1-8b38-4f6e-822b-28a34a8fce22 None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:41.940127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 644/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Fri Jun 24 04:02:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:41.946190 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f053862-db5c-4b83-bc9e-64e33891f03b None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:41.977400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:42.044069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f053862-db5c-4b83-bc9e-64e33891f03b None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:42.079021 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88da0f-27b8-4b52-b53e-4b11d86f391f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:42.079667 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88da0f-27b8-4b52-b53e-4b11d86f391f None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:42.079950 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88da0f-27b8-4b52-b53e-4b11d86f391f None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:42.080226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88da0f-27b8-4b52-b53e-4b11d86f391f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:42.081012 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88da0f-27b8-4b52-b53e-4b11d86f391f None admin] RBAC: Adding query filter params (interface=public, service_id=019bd9578090404e8627704f650c934f, region_id=RegionOne) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:42.081323 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88da0f-27b8-4b52-b53e-4b11d86f391f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:42.082757 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88da0f-27b8-4b52-b53e-4b11d86f391f None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:42.087740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 656/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Jun 24 04:02:41 2022] GET /identity/v3/endpoints?service_id=019bd9578090404e8627704f650c934f&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:42.093385 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa689c1-8b38-4f6e-822b-28a34a8fce22 None admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:42.181433 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:42.199618 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa689c1-8b38-4f6e-822b-28a34a8fce22 None admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:42.232818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28d522-4f0b-48d4-b130-d8f1cc4fc86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:42.233502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28d522-4f0b-48d4-b130-d8f1cc4fc86c None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:42.233766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28d522-4f0b-48d4-b130-d8f1cc4fc86c None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:42.234049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28d522-4f0b-48d4-b130-d8f1cc4fc86c None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:42.235383 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28d522-4f0b-48d4-b130-d8f1cc4fc86c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:42.236762 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28d522-4f0b-48d4-b130-d8f1cc4fc86c None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:42.243502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 645/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Jun 24 04:02:42 2022] GET /identity/v3/services => generated 2191 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:02:44.284567 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1641d1-01bc-4b50-b374-d2886bbbb52a None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:44.285093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1641d1-01bc-4b50-b374-d2886bbbb52a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:44.285204 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1641d1-01bc-4b50-b374-d2886bbbb52a None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:44.286574 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 657/1302] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:02:44.291809 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba9576-c4d1-4e85-99bc-a38b2bb7029d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:44.292319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba9576-c4d1-4e85-99bc-a38b2bb7029d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:44.292422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba9576-c4d1-4e85-99bc-a38b2bb7029d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:44.309592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c3ba9576-c4d1-4e85-99bc-a38b2bb7029d None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:02:44.310823 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c3ba9576-c4d1-4e85-99bc-a38b2bb7029d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:02:44.377813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 646/1303] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:02:44 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 04:02:44.401514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1641d1-01bc-4b50-b374-d2886bbbb52a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:44.496375 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:44.504075 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1641d1-01bc-4b50-b374-d2886bbbb52a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:44.536668 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cdef19-e2ff-41e4-8218-b828a5a7164c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:44.537472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdef19-e2ff-41e4-8218-b828a5a7164c None admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:44.538010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdef19-e2ff-41e4-8218-b828a5a7164c None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:44.538464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdef19-e2ff-41e4-8218-b828a5a7164c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:44.539496 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cdef19-e2ff-41e4-8218-b828a5a7164c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:44.540861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cdef19-e2ff-41e4-8218-b828a5a7164c None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:44.553559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application [None req-35cdef19-e2ff-41e4-8218-b828a5a7164c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 04:02:44.553559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:44.553559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 04:02:44.553559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 04:02:44.553559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 04:02:44.553559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 04:02:44.553559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 04:02:44.553559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 04:02:44.553559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 04:02:44.553559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:44.553559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 04:02:44.553559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:44.553559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 04:02:44.553559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 04:02:44.553559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 04:02:44.553559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 04:02:44.553559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 04:02:44.554794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 04:02:44.554794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 04:02:44.554794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 04:02:44.554794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 04:02:44.554794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 04:02:44.554794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 04:02:44.554794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 04:02:44.554794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 04:02:44.554794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 04:02:44.554794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 04:02:44.554794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 04:02:44.554794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 04:02:44.554794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 04:02:44.554794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 04:02:44.554794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 04:02:44.554794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 04:02:44.555803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 04:02:44.555803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application with Lock( Jun 24 04:02:44.555803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 04:02:44.555803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self._enter() Jun 24 04:02:44.555803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 04:02:44.555803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 04:02:44.555803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 04:02:44.555803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application return self.creator() Jun 24 04:02:44.555803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 04:02:44.555803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application created_value = creator( Jun 24 04:02:44.555803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 04:02:44.555803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 04:02:44.555803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 04:02:44.555803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: ERROR keystone.server.flask.application Jun 24 04:02:44.555803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 658/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Jun 24 04:02:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 04:02:44.561025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba9576-c4d1-4e85-99bc-a38b2bb7029d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:44.637107 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:44.660033 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba9576-c4d1-4e85-99bc-a38b2bb7029d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:44.693655 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebe3ce-cad8-4445-8062-2c81ec3e11bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:44.694578 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebe3ce-cad8-4445-8062-2c81ec3e11bd None admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:44.695244 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebe3ce-cad8-4445-8062-2c81ec3e11bd None admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:44.695526 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebe3ce-cad8-4445-8062-2c81ec3e11bd None admin] PATH_INFO: `/v3/services` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:44.696235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bebe3ce-cad8-4445-8062-2c81ec3e11bd None admin] RBAC: Adding query filter params (name=placement) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:02:44.696546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bebe3ce-cad8-4445-8062-2c81ec3e11bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:44.697966 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bebe3ce-cad8-4445-8062-2c81ec3e11bd None admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:44.703228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 647/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Fri Jun 24 04:02:44 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:02:44.708928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cdef19-e2ff-41e4-8218-b828a5a7164c None admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:02:44.808054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cdef19-e2ff-41e4-8218-b828a5a7164c None admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:02:44.839902 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb32a0-f95b-4788-8e63-11c026fcb9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:02:44.840550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb32a0-f95b-4788-8e63-11c026fcb9d9 None admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:02:44.840836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb32a0-f95b-4788-8e63-11c026fcb9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:02:44.841114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb32a0-f95b-4788-8e63-11c026fcb9d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:02:44.841847 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb32a0-f95b-4788-8e63-11c026fcb9d9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:02:44.843338 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb32a0-f95b-4788-8e63-11c026fcb9d9 None admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:02:44.929315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 659/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Jun 24 04:02:44 2022] POST /identity/v3/endpoints => generated 331 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:02:54.699983 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=30 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:02:54.924748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:01.118621 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad57619-d59d-4327-8a73-f849f8c44f9a None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:01.119216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad57619-d59d-4327-8a73-f849f8c44f9a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:01.119216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad57619-d59d-4327-8a73-f849f8c44f9a None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:01.127734 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d5ebb-542a-4831-9f9d-ea3c8b2baff8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:01.128205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d5ebb-542a-4831-9f9d-ea3c8b2baff8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:01.128480 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d5ebb-542a-4831-9f9d-ea3c8b2baff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:01.130456 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 648/1307] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:09:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:01.209677 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-606d5ebb-542a-4831-9f9d-ea3c8b2baff8 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:01.211148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-606d5ebb-542a-4831-9f9d-ea3c8b2baff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:01.339002 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 660/1308] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:09:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:01.380013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca710e06-3eea-46ab-9be8-7c95bef3c4cc None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:01.380310 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca710e06-3eea-46ab-9be8-7c95bef3c4cc None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:01.380657 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca710e06-3eea-46ab-9be8-7c95bef3c4cc None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:01.382012 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 649/1309] 10.222.0.16 () {54 vars in 964 bytes} [Fri Jun 24 04:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:01.390323 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e534a2d-5224-4310-bba7-c4f207da30db None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:01.390659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e534a2d-5224-4310-bba7-c4f207da30db None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:01.390950 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e534a2d-5224-4310-bba7-c4f207da30db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:01.493110 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8e534a2d-5224-4310-bba7-c4f207da30db None None] MFA Rules not processed for user `fd98b54f587f4b42b4dd0a058b30c684`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:01.494390 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8e534a2d-5224-4310-bba7-c4f207da30db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:01.516353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:01.577443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 661/1310] 10.222.0.16 () {58 vars in 1061 bytes} [Fri Jun 24 04:09:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:01.586050 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca710e06-3eea-46ab-9be8-7c95bef3c4cc None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:09:01.587734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ca710e06-3eea-46ab-9be8-7c95bef3c4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:01.734154 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca710e06-3eea-46ab-9be8-7c95bef3c4cc None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:09:01.815435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895787d-0c6f-443d-ac60-61557d7a5a44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:09:01.816087 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895787d-0c6f-443d-ac60-61557d7a5a44 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:01.816371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895787d-0c6f-443d-ac60-61557d7a5a44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:01.816682 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895787d-0c6f-443d-ac60-61557d7a5a44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:01.818303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d895787d-0c6f-443d-ac60-61557d7a5a44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:01.877073 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:01.932836 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895787d-0c6f-443d-ac60-61557d7a5a44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:09:01.934282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895787d-0c6f-443d-ac60-61557d7a5a44 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:09:01.954001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 650/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:09:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:09:04.324328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f97e9-a0a2-4317-9165-a1ec39df094c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:04.324838 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f97e9-a0a2-4317-9165-a1ec39df094c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:04.324838 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60f97e9-a0a2-4317-9165-a1ec39df094c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:04.326303 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 662/1312] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:04.331769 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445a6d3-8c96-4548-8ae3-3eb927826088 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:04.332190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445a6d3-8c96-4548-8ae3-3eb927826088 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:04.332327 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445a6d3-8c96-4548-8ae3-3eb927826088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:04.414718 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f445a6d3-8c96-4548-8ae3-3eb927826088 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:04.415943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f445a6d3-8c96-4548-8ae3-3eb927826088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:04.527924 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 651/1313] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:09:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:04.566834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f97e9-a0a2-4317-9165-a1ec39df094c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:09:04.568371 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e60f97e9-a0a2-4317-9165-a1ec39df094c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:04.686492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60f97e9-a0a2-4317-9165-a1ec39df094c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:09:04.766621 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:04.768318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424b094-a766-411a-a81b-d9f474e8f101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:09:04.769284 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424b094-a766-411a-a81b-d9f474e8f101 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:04.769633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424b094-a766-411a-a81b-d9f474e8f101 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:04.769973 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424b094-a766-411a-a81b-d9f474e8f101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:04.771730 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2424b094-a766-411a-a81b-d9f474e8f101 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:04.894635 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2424b094-a766-411a-a81b-d9f474e8f101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:09:04.896546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2424b094-a766-411a-a81b-d9f474e8f101 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:09:04.916979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 663/1314] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:09:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:09:07.507460 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6986c1b-b9e5-40f6-8164-ab8852a40615 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:07.508034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6986c1b-b9e5-40f6-8164-ab8852a40615 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:07.508034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6986c1b-b9e5-40f6-8164-ab8852a40615 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:07.509411 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 652/1315] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:07.514708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b770c-5fc1-4ec5-81c5-59788c588dab None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:07.515133 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b770c-5fc1-4ec5-81c5-59788c588dab None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:07.515285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b770c-5fc1-4ec5-81c5-59788c588dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:07.578260 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b49b770c-5fc1-4ec5-81c5-59788c588dab None None] MFA Rules not processed for user `1b3131ee9b304a72b169fab71c8af3d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:07.579492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b49b770c-5fc1-4ec5-81c5-59788c588dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:07.631226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:07.676723 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 664/1316] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:09:07 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:07.716937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6986c1b-b9e5-40f6-8164-ab8852a40615 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:09:07.718498 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f6986c1b-b9e5-40f6-8164-ab8852a40615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:07.728223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:07.821191 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6986c1b-b9e5-40f6-8164-ab8852a40615 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:09:07.892909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5468f63-33aa-4a57-9420-0e09cbfed36a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:09:07.893553 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5468f63-33aa-4a57-9420-0e09cbfed36a service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:07.893838 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5468f63-33aa-4a57-9420-0e09cbfed36a service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:07.894115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5468f63-33aa-4a57-9420-0e09cbfed36a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:07.895777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e5468f63-33aa-4a57-9420-0e09cbfed36a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:07.957260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:08.004329 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5468f63-33aa-4a57-9420-0e09cbfed36a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:09:08.005786 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5468f63-33aa-4a57-9420-0e09cbfed36a service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:09:08.025805 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 653/1317] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:09:07 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:09:10.811109 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f171166-08eb-4ceb-9968-981f48b1430a None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:10.811777 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f171166-08eb-4ceb-9968-981f48b1430a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:10.811777 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f171166-08eb-4ceb-9968-981f48b1430a None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:10.813080 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 665/1318] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:10.818497 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba772c92-d8f0-4037-9f20-921e42153bdb None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:10.818950 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba772c92-d8f0-4037-9f20-921e42153bdb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:10.819052 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba772c92-d8f0-4037-9f20-921e42153bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:10.895738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ba772c92-d8f0-4037-9f20-921e42153bdb None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:10.896937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ba772c92-d8f0-4037-9f20-921e42153bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:10.990850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 654/1319] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:09:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:11.030553 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f171166-08eb-4ceb-9968-981f48b1430a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:09:11.032080 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3f171166-08eb-4ceb-9968-981f48b1430a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:11.140830 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f171166-08eb-4ceb-9968-981f48b1430a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:09:11.236452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9554c1-6db8-4084-bb6f-9dff1d9e5b5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:09:11.237105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9554c1-6db8-4084-bb6f-9dff1d9e5b5d service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:11.237525 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9554c1-6db8-4084-bb6f-9dff1d9e5b5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:11.237669 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9554c1-6db8-4084-bb6f-9dff1d9e5b5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:11.239317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3e9554c1-6db8-4084-bb6f-9dff1d9e5b5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:11.246861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:11.347084 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9554c1-6db8-4084-bb6f-9dff1d9e5b5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:09:11.348516 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9554c1-6db8-4084-bb6f-9dff1d9e5b5d service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:09:11.367228 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 666/1320] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:09:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:09:14.500223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7633e8-050b-4c95-b402-30589515c812 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:14.500805 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7633e8-050b-4c95-b402-30589515c812 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:14.500805 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7633e8-050b-4c95-b402-30589515c812 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:14.502166 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 655/1321] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:14.507418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2f079-333d-4169-8bdc-53ab923fdded None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:14.507896 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2f079-333d-4169-8bdc-53ab923fdded None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:14.508099 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2f079-333d-4169-8bdc-53ab923fdded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:14.576297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0cc2f079-333d-4169-8bdc-53ab923fdded None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:14.577546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0cc2f079-333d-4169-8bdc-53ab923fdded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:14.641903 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:14.684491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 667/1322] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:14.721028 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7633e8-050b-4c95-b402-30589515c812 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:09:14.722698 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-eb7633e8-050b-4c95-b402-30589515c812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:14.742765 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:14.833730 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7633e8-050b-4c95-b402-30589515c812 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:09:14.909490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1f645-3ad3-496f-9449-bf953959582d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:09:14.910142 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1f645-3ad3-496f-9449-bf953959582d service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:14.910502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1f645-3ad3-496f-9449-bf953959582d service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:14.910805 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1f645-3ad3-496f-9449-bf953959582d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:14.912461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-11b1f645-3ad3-496f-9449-bf953959582d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:14.984135 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:15.021891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1f645-3ad3-496f-9449-bf953959582d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:09:15.023372 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1f645-3ad3-496f-9449-bf953959582d service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:09:15.044336 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 656/1323] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:09:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:09:17.190687 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db3b1b-14ad-4991-92ee-7b629ac1c90c None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:17.191280 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db3b1b-14ad-4991-92ee-7b629ac1c90c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:17.191280 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db3b1b-14ad-4991-92ee-7b629ac1c90c None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:17.192699 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 668/1324] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:17.198502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb416eee-444a-4410-bc99-66b99e7496d5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:17.199024 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb416eee-444a-4410-bc99-66b99e7496d5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:17.199170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb416eee-444a-4410-bc99-66b99e7496d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:17.267109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-eb416eee-444a-4410-bc99-66b99e7496d5 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:17.268324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-eb416eee-444a-4410-bc99-66b99e7496d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:17.372476 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 657/1325] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:17.424322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974621be-ac0d-4794-834a-71f3fcab8291 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:17.424605 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974621be-ac0d-4794-834a-71f3fcab8291 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:17.424879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974621be-ac0d-4794-834a-71f3fcab8291 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:17.426261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 669/1326] 10.222.0.16 () {54 vars in 964 bytes} [Fri Jun 24 04:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:17.432668 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74f4f7-3015-45d3-8de2-fef122c2b1c3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:17.432963 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74f4f7-3015-45d3-8de2-fef122c2b1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:17.433246 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74f4f7-3015-45d3-8de2-fef122c2b1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:17.463784 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:17.515603 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0d74f4f7-3015-45d3-8de2-fef122c2b1c3 None None] MFA Rules not processed for user `fd98b54f587f4b42b4dd0a058b30c684`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:17.516838 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0d74f4f7-3015-45d3-8de2-fef122c2b1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:17.593073 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 658/1327] 10.222.0.16 () {58 vars in 1061 bytes} [Fri Jun 24 04:09:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:17.602147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974621be-ac0d-4794-834a-71f3fcab8291 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:09:17.603909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-974621be-ac0d-4794-834a-71f3fcab8291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:17.715241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974621be-ac0d-4794-834a-71f3fcab8291 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:09:17.788988 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ace014-4d3c-45f1-82c2-480061eefd9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:09:17.789648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ace014-4d3c-45f1-82c2-480061eefd9d service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:17.789933 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ace014-4d3c-45f1-82c2-480061eefd9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:17.790211 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ace014-4d3c-45f1-82c2-480061eefd9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:17.792562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b5ace014-4d3c-45f1-82c2-480061eefd9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:17.809435 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:17.902564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ace014-4d3c-45f1-82c2-480061eefd9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:09:17.904003 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ace014-4d3c-45f1-82c2-480061eefd9d service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:09:17.922525 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 670/1328] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:09:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:09:20.258560 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f6e60-d98f-47b0-a360-2b8a8b91fc3c None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:20.259106 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f6e60-d98f-47b0-a360-2b8a8b91fc3c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:20.259106 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f6e60-d98f-47b0-a360-2b8a8b91fc3c None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:20.260734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 659/1329] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:09:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:20.266048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b07edc-096e-4084-ad00-6dad511e1e73 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:20.266468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b07edc-096e-4084-ad00-6dad511e1e73 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:20.266648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b07edc-096e-4084-ad00-6dad511e1e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:20.357715 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-74b07edc-096e-4084-ad00-6dad511e1e73 None None] MFA Rules not processed for user `1b3131ee9b304a72b169fab71c8af3d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:20.358980 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-74b07edc-096e-4084-ad00-6dad511e1e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:20.432591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:20.465792 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 671/1330] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:09:20 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:20.503774 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f6e60-d98f-47b0-a360-2b8a8b91fc3c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:09:20.505305 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bc6f6e60-d98f-47b0-a360-2b8a8b91fc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:20.613044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f6e60-d98f-47b0-a360-2b8a8b91fc3c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:09:20.634356 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:20.690660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901fa29-490b-481e-8269-d7af986bea5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:09:20.691309 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901fa29-490b-481e-8269-d7af986bea5c service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:20.691590 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901fa29-490b-481e-8269-d7af986bea5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:20.691874 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901fa29-490b-481e-8269-d7af986bea5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:20.693499 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7901fa29-490b-481e-8269-d7af986bea5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:20.802629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901fa29-490b-481e-8269-d7af986bea5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:09:20.804102 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901fa29-490b-481e-8269-d7af986bea5c service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:09:20.821899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 660/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:09:20 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:09:23.415942 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f0b3a-c27d-4c65-bf4c-36afb1c740d0 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:23.416479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f0b3a-c27d-4c65-bf4c-36afb1c740d0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:23.416479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f0b3a-c27d-4c65-bf4c-36afb1c740d0 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:23.417883 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 672/1332] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:09:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:23.423133 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0584b-7792-4fa8-9172-fb2ecfc8df35 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:23.423564 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0584b-7792-4fa8-9172-fb2ecfc8df35 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:23.423702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0584b-7792-4fa8-9172-fb2ecfc8df35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:23.486573 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:23.489190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-04a0584b-7792-4fa8-9172-fb2ecfc8df35 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:23.490448 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-04a0584b-7792-4fa8-9172-fb2ecfc8df35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:23.586067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 661/1333] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:09:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:23.622416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f0b3a-c27d-4c65-bf4c-36afb1c740d0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:09:23.623939 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-610f0b3a-c27d-4c65-bf4c-36afb1c740d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:23.725714 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f0b3a-c27d-4c65-bf4c-36afb1c740d0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:09:23.797262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32ea27-f979-49b9-9e3d-8a0f25dfdab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:09:23.797896 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32ea27-f979-49b9-9e3d-8a0f25dfdab7 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:23.798176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32ea27-f979-49b9-9e3d-8a0f25dfdab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:23.798482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32ea27-f979-49b9-9e3d-8a0f25dfdab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:23.800101 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7a32ea27-f979-49b9-9e3d-8a0f25dfdab7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:23.826709 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:23.919255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32ea27-f979-49b9-9e3d-8a0f25dfdab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:09:23.920696 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32ea27-f979-49b9-9e3d-8a0f25dfdab7 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:09:23.938765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 673/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:09:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:09:26.036698 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca03fb-3d9a-4a24-9335-2e2782a80ab3 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:26.037261 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca03fb-3d9a-4a24-9335-2e2782a80ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:26.037261 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca03fb-3d9a-4a24-9335-2e2782a80ab3 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:26.038717 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 662/1335] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:26.044632 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507904d-e08c-4b95-8dca-3c9b10148352 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:26.045067 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507904d-e08c-4b95-8dca-3c9b10148352 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:26.045199 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507904d-e08c-4b95-8dca-3c9b10148352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:26.109510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9507904d-e08c-4b95-8dca-3c9b10148352 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:26.110732 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9507904d-e08c-4b95-8dca-3c9b10148352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:26.194076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:26.217852 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 674/1336] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:09:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:26.258792 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca03fb-3d9a-4a24-9335-2e2782a80ab3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:09:26.261405 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e0ca03fb-3d9a-4a24-9335-2e2782a80ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:26.366090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca03fb-3d9a-4a24-9335-2e2782a80ab3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:09:26.395124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:26.440546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf8014-e236-43a3-b873-dde8c77af522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:09:26.441193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf8014-e236-43a3-b873-dde8c77af522 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:26.441478 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf8014-e236-43a3-b873-dde8c77af522 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:26.441755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf8014-e236-43a3-b873-dde8c77af522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:26.446976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9adf8014-e236-43a3-b873-dde8c77af522 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:26.568415 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf8014-e236-43a3-b873-dde8c77af522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:09:26.569862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf8014-e236-43a3-b873-dde8c77af522 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:09:26.594094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 663/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:09:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:09:29.270254 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60188eea-db76-449b-97c1-78222d907ef0 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:29.270829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60188eea-db76-449b-97c1-78222d907ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:29.270829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60188eea-db76-449b-97c1-78222d907ef0 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:29.272153 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 675/1338] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:29.277771 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721492e4-decc-4680-bb9c-3323c3f25fe1 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:29.278175 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721492e4-decc-4680-bb9c-3323c3f25fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:29.278334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721492e4-decc-4680-bb9c-3323c3f25fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:29.343599 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-721492e4-decc-4680-bb9c-3323c3f25fe1 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:29.344799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-721492e4-decc-4680-bb9c-3323c3f25fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:29.353336 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:29.443680 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 664/1339] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:09:29 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:29.479550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60188eea-db76-449b-97c1-78222d907ef0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:09:29.481092 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-60188eea-db76-449b-97c1-78222d907ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:29.584398 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60188eea-db76-449b-97c1-78222d907ef0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:09:29.655527 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83afb2d3-b1fc-44d8-afed-47cd9199c1bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:09:29.656169 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afb2d3-b1fc-44d8-afed-47cd9199c1bb service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:29.656455 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afb2d3-b1fc-44d8-afed-47cd9199c1bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:29.656732 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83afb2d3-b1fc-44d8-afed-47cd9199c1bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:29.658351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-83afb2d3-b1fc-44d8-afed-47cd9199c1bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:29.693656 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:29.767229 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afb2d3-b1fc-44d8-afed-47cd9199c1bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:09:29.768662 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83afb2d3-b1fc-44d8-afed-47cd9199c1bb service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:09:29.789967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 676/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:09:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:09:33.994853 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96442ed5-25fb-4633-95db-fb8da0df2b21 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:33.995348 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96442ed5-25fb-4633-95db-fb8da0df2b21 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:33.995475 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96442ed5-25fb-4633-95db-fb8da0df2b21 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:33.997209 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 665/1341] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:34.002686 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb034ca-6743-4c6a-b7e5-078dfb013693 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:34.003138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb034ca-6743-4c6a-b7e5-078dfb013693 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:34.003246 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb034ca-6743-4c6a-b7e5-078dfb013693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:34.088458 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8bb034ca-6743-4c6a-b7e5-078dfb013693 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:34.089683 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8bb034ca-6743-4c6a-b7e5-078dfb013693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:34.186425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:34.202402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 677/1342] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:09:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:34.238221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96442ed5-25fb-4633-95db-fb8da0df2b21 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:09:34.240368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-96442ed5-25fb-4633-95db-fb8da0df2b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:34.338289 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96442ed5-25fb-4633-95db-fb8da0df2b21 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:09:34.341793 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6787e9b8-ff39-450a-a579-e8b12653eb41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:09:34.342467 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787e9b8-ff39-450a-a579-e8b12653eb41 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:34.342755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787e9b8-ff39-450a-a579-e8b12653eb41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:34.343029 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787e9b8-ff39-450a-a579-e8b12653eb41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:34.351922 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6787e9b8-ff39-450a-a579-e8b12653eb41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:09:34.353478 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6787e9b8-ff39-450a-a579-e8b12653eb41 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:09:34.362949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 666/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:09:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:09:37.668104 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1edb8cc-2b93-4bb9-8f75-a667b381a4d8 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:37.668700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1edb8cc-2b93-4bb9-8f75-a667b381a4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:37.668700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1edb8cc-2b93-4bb9-8f75-a667b381a4d8 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:37.670124 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 678/1344] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 04:09:37.675666 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b837b-9437-4b19-8a0a-c345ae188374 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:37.676081 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b837b-9437-4b19-8a0a-c345ae188374 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:37.676238 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b837b-9437-4b19-8a0a-c345ae188374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:37.704864 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-958b837b-9437-4b19-8a0a-c345ae188374 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:37.706067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-958b837b-9437-4b19-8a0a-c345ae188374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:37.735020 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:37.784161 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 667/1345] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:09:37 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:37.822669 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1edb8cc-2b93-4bb9-8f75-a667b381a4d8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:09:37.824419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d1edb8cc-2b93-4bb9-8f75-a667b381a4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:37.837222 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1edb8cc-2b93-4bb9-8f75-a667b381a4d8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:09:37.840708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b74192-8220-4fe3-938d-1c6286aa6ac2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:09:37.841355 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b74192-8220-4fe3-938d-1c6286aa6ac2 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:37.841648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b74192-8220-4fe3-938d-1c6286aa6ac2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:37.841933 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b74192-8220-4fe3-938d-1c6286aa6ac2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:37.850983 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b74192-8220-4fe3-938d-1c6286aa6ac2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:09:37.852963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b74192-8220-4fe3-938d-1c6286aa6ac2 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:09:37.862438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 679/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:09:37 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:09:42.382075 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06660497-5816-4bd9-8881-614af3caaaa7 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:42.382710 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06660497-5816-4bd9-8881-614af3caaaa7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:42.382994 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06660497-5816-4bd9-8881-614af3caaaa7 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:42.385083 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 668/1347] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:09:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:42.390680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb6646-f8a7-4f85-a4a4-8a4c41e64b78 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:42.391201 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb6646-f8a7-4f85-a4a4-8a4c41e64b78 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:42.391306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb6646-f8a7-4f85-a4a4-8a4c41e64b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:42.409354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0fdb6646-f8a7-4f85-a4a4-8a4c41e64b78 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:42.410611 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0fdb6646-f8a7-4f85-a4a4-8a4c41e64b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:42.419304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 680/1348] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:09:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:42.458569 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06660497-5816-4bd9-8881-614af3caaaa7 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:09:42.470026 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06660497-5816-4bd9-8881-614af3caaaa7 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:09:42.473661 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e46c3-74a3-4899-b6d6-b82dfd3c0af2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:09:42.474310 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e46c3-74a3-4899-b6d6-b82dfd3c0af2 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:42.474628 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e46c3-74a3-4899-b6d6-b82dfd3c0af2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:42.474903 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e46c3-74a3-4899-b6d6-b82dfd3c0af2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:42.485268 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e46c3-74a3-4899-b6d6-b82dfd3c0af2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:09:42.486751 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e46c3-74a3-4899-b6d6-b82dfd3c0af2 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:09:42.497937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 669/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:09:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:09:47.497564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9e042-fe47-4c55-a890-2daebd0cd05a None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:47.498548 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9e042-fe47-4c55-a890-2daebd0cd05a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:47.498847 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9e042-fe47-4c55-a890-2daebd0cd05a None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:47.500855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 681/1350] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jun 24 04:09:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:47.505920 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b403292-3a28-437a-8d38-e60500ccc450 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:47.506969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b403292-3a28-437a-8d38-e60500ccc450 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:47.507459 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b403292-3a28-437a-8d38-e60500ccc450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:47.636229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-5b403292-3a28-437a-8d38-e60500ccc450 None None] MFA Rules not processed for user `7662e61619d44c92af3ffacbacba3913`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:47.637822 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5b403292-3a28-437a-8d38-e60500ccc450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:47.717231 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 670/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jun 24 04:09:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:51.940811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ef365-b7a6-4016-95e6-9a01f5deafaa None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:51.941331 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ef365-b7a6-4016-95e6-9a01f5deafaa None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:51.941613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ef365-b7a6-4016-95e6-9a01f5deafaa None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:51.943063 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 682/1352] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jun 24 04:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:51.948416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c66a6-0d15-4d2f-9742-16df2090fe80 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:51.948883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c66a6-0d15-4d2f-9742-16df2090fe80 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:51.949218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93c66a6-0d15-4d2f-9742-16df2090fe80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:51.967871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b93c66a6-0d15-4d2f-9742-16df2090fe80 None None] MFA Rules not processed for user `7662e61619d44c92af3ffacbacba3913`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:51.969175 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b93c66a6-0d15-4d2f-9742-16df2090fe80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:51.978584 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 671/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jun 24 04:09:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:52.406822 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=13 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:53.402742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5e3bf-0547-44a0-b151-90f892adda2e None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:53.403405 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5e3bf-0547-44a0-b151-90f892adda2e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:53.403405 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5e3bf-0547-44a0-b151-90f892adda2e None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:53.404672 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 683/1354] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jun 24 04:09:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:53.409898 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7103eca-a98d-478c-9f46-2de68c25cdc4 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:53.410314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7103eca-a98d-478c-9f46-2de68c25cdc4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:53.410757 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7103eca-a98d-478c-9f46-2de68c25cdc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:53.428993 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c7103eca-a98d-478c-9f46-2de68c25cdc4 None None] MFA Rules not processed for user `7662e61619d44c92af3ffacbacba3913`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:53.430256 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c7103eca-a98d-478c-9f46-2de68c25cdc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:53.438974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 672/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jun 24 04:09:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:57.213958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef0175-e836-4ebb-a683-9d1287868d00 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:57.214820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef0175-e836-4ebb-a683-9d1287868d00 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:57.215245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef0175-e836-4ebb-a683-9d1287868d00 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:57.218517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 684/1356] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:09:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:57.224307 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b29ba-d1d7-4774-a73f-c444764a1f28 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:57.225139 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b29ba-d1d7-4774-a73f-c444764a1f28 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:57.225577 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b29ba-d1d7-4774-a73f-c444764a1f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:57.244991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4d8b29ba-d1d7-4774-a73f-c444764a1f28 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:57.246190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4d8b29ba-d1d7-4774-a73f-c444764a1f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:57.254320 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 673/1357] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:09:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:57.321786 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f92dd1-0c6a-4e83-857d-fd303f2178d6 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:57.322282 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f92dd1-0c6a-4e83-857d-fd303f2178d6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:57.322742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f92dd1-0c6a-4e83-857d-fd303f2178d6 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:57.324586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 685/1358] 10.222.0.16 () {54 vars in 961 bytes} [Fri Jun 24 04:09:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:57.333730 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb963c1-252d-4da7-a038-5891e3fb5622 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:57.334212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb963c1-252d-4da7-a038-5891e3fb5622 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:57.334681 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb963c1-252d-4da7-a038-5891e3fb5622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:57.386230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:09:57.395489 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-dcb963c1-252d-4da7-a038-5891e3fb5622 None None] MFA Rules not processed for user `80b7b3ec60db4de19bbf7d08f111f470`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:57.397065 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dcb963c1-252d-4da7-a038-5891e3fb5622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:57.483910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 674/1359] 10.222.0.16 () {58 vars in 1058 bytes} [Fri Jun 24 04:09:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:09:57.490989 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f92dd1-0c6a-4e83-857d-fd303f2178d6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:09:57.507632 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f92dd1-0c6a-4e83-857d-fd303f2178d6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:09:57.514185 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383671a-9567-4648-99aa-526e6b82dd91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:09:57.514953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383671a-9567-4648-99aa-526e6b82dd91 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:57.515303 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383671a-9567-4648-99aa-526e6b82dd91 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:57.515657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383671a-9567-4648-99aa-526e6b82dd91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:57.525189 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383671a-9567-4648-99aa-526e6b82dd91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:09:57.526895 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383671a-9567-4648-99aa-526e6b82dd91 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:09:57.537148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 686/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:09:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:09:59.818756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626610ad-83e5-4ae5-b4c8-12f9d19962d9 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:59.819279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626610ad-83e5-4ae5-b4c8-12f9d19962d9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:59.819407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626610ad-83e5-4ae5-b4c8-12f9d19962d9 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:59.821267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 675/1361] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:09:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:09:59.826546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd79b67-a4ca-4eef-b625-4f58b9a510fb None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:59.826964 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd79b67-a4ca-4eef-b625-4f58b9a510fb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:59.827105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd79b67-a4ca-4eef-b625-4f58b9a510fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:59.846972 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6cd79b67-a4ca-4eef-b625-4f58b9a510fb None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:09:59.848306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6cd79b67-a4ca-4eef-b625-4f58b9a510fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:09:59.857372 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 687/1362] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:09:59.902522 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626610ad-83e5-4ae5-b4c8-12f9d19962d9 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:09:59.917064 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626610ad-83e5-4ae5-b4c8-12f9d19962d9 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:09:59.922484 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4aa0ab-b5b7-46e9-95c9-6fd4f8645fc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:09:59.923126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4aa0ab-b5b7-46e9-95c9-6fd4f8645fc1 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:09:59.923412 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4aa0ab-b5b7-46e9-95c9-6fd4f8645fc1 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:09:59.923690 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4aa0ab-b5b7-46e9-95c9-6fd4f8645fc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:09:59.933206 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4aa0ab-b5b7-46e9-95c9-6fd4f8645fc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:09:59.934850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4aa0ab-b5b7-46e9-95c9-6fd4f8645fc1 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:09:59.944474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 676/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:09:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:02.495073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa5081-28ba-4b45-8324-c6bd15f70305 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:02.495584 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa5081-28ba-4b45-8324-c6bd15f70305 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:02.495719 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaa5081-28ba-4b45-8324-c6bd15f70305 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:02.497191 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 688/1364] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:02.502821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769ef98-5fa6-47c0-8283-94a191ea584b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:02.503258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769ef98-5fa6-47c0-8283-94a191ea584b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:02.503517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769ef98-5fa6-47c0-8283-94a191ea584b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:02.521869 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f769ef98-5fa6-47c0-8283-94a191ea584b None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:02.523097 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f769ef98-5fa6-47c0-8283-94a191ea584b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:02.531577 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 677/1365] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:10:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:02.577239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382f598-9f2f-48cd-8370-44be831ef3d3 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:02.577527 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382f598-9f2f-48cd-8370-44be831ef3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:02.577802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382f598-9f2f-48cd-8370-44be831ef3d3 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:02.579185 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 689/1366] 10.222.0.16 () {54 vars in 961 bytes} [Fri Jun 24 04:10:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:02.584523 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e7e8d-15b7-4b81-b45c-78ad21a0f984 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:02.584824 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e7e8d-15b7-4b81-b45c-78ad21a0f984 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:02.588279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e7e8d-15b7-4b81-b45c-78ad21a0f984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:02.603297 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-eb9e7e8d-15b7-4b81-b45c-78ad21a0f984 None None] MFA Rules not processed for user `80b7b3ec60db4de19bbf7d08f111f470`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:02.604492 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-eb9e7e8d-15b7-4b81-b45c-78ad21a0f984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:02.613722 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 678/1367] 10.222.0.16 () {58 vars in 1058 bytes} [Fri Jun 24 04:10:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:02.621348 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382f598-9f2f-48cd-8370-44be831ef3d3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:02.637176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382f598-9f2f-48cd-8370-44be831ef3d3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:02.642734 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714860cc-991f-453f-b5e1-6a91ff50eb62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:02.643388 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714860cc-991f-453f-b5e1-6a91ff50eb62 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:02.643677 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714860cc-991f-453f-b5e1-6a91ff50eb62 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:02.643960 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714860cc-991f-453f-b5e1-6a91ff50eb62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:02.653262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714860cc-991f-453f-b5e1-6a91ff50eb62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:02.654900 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714860cc-991f-453f-b5e1-6a91ff50eb62 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:02.664567 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 690/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:04.960585 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52ed6d-36f3-4737-af51-834da16ef427 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:04.961278 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52ed6d-36f3-4737-af51-834da16ef427 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:04.961278 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52ed6d-36f3-4737-af51-834da16ef427 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:04.962831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 679/1369] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:04.968016 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd931c-6b51-41e6-b257-93d6acbfa8e8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:04.968472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd931c-6b51-41e6-b257-93d6acbfa8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:04.968769 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd931c-6b51-41e6-b257-93d6acbfa8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:04.986502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-81cd931c-6b51-41e6-b257-93d6acbfa8e8 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:04.987790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-81cd931c-6b51-41e6-b257-93d6acbfa8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:04.996334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 691/1370] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:10:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:05.041527 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52ed6d-36f3-4737-af51-834da16ef427 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:05.058453 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f52ed6d-36f3-4737-af51-834da16ef427 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:05.064126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38830bc0-73c8-41c6-8761-37ff87b92d27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:05.064771 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38830bc0-73c8-41c6-8761-37ff87b92d27 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:05.065055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38830bc0-73c8-41c6-8761-37ff87b92d27 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:05.065332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38830bc0-73c8-41c6-8761-37ff87b92d27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:05.074450 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38830bc0-73c8-41c6-8761-37ff87b92d27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:05.076064 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38830bc0-73c8-41c6-8761-37ff87b92d27 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:05.087071 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 680/1371] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:05.525565 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58b54b-3d3e-48b3-8856-3a9480fc2e62 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:05.525790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58b54b-3d3e-48b3-8856-3a9480fc2e62 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:05.526264 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb58b54b-3d3e-48b3-8856-3a9480fc2e62 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:05.527684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 692/1372] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:10:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:05.532980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbe477-23f1-41aa-944a-c36aec5ffe1c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:05.533274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbe477-23f1-41aa-944a-c36aec5ffe1c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:05.533559 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dbe477-23f1-41aa-944a-c36aec5ffe1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:05.553279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-24dbe477-23f1-41aa-944a-c36aec5ffe1c None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:05.555632 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-24dbe477-23f1-41aa-944a-c36aec5ffe1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:05.563552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 681/1373] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:10:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:05.883069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196dba5d-5e73-45ee-aac7-3fccffc7f149 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:05.884085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196dba5d-5e73-45ee-aac7-3fccffc7f149 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:05.884376 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196dba5d-5e73-45ee-aac7-3fccffc7f149 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:05.885738 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 693/1374] 10.222.0.16 () {54 vars in 963 bytes} [Fri Jun 24 04:10:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:05.890843 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc66a0-e51e-47cc-b475-42fe2d38ab1f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:05.891130 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc66a0-e51e-47cc-b475-42fe2d38ab1f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:05.891408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc66a0-e51e-47cc-b475-42fe2d38ab1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:05.943195 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-31fc66a0-e51e-47cc-b475-42fe2d38ab1f None None] MFA Rules not processed for user `0fbec7435e0841a5b96946dca814e0c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:05.944395 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-31fc66a0-e51e-47cc-b475-42fe2d38ab1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:06.009509 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 682/1375] 10.222.0.16 () {58 vars in 1060 bytes} [Fri Jun 24 04:10:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:10:06.017595 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196dba5d-5e73-45ee-aac7-3fccffc7f149 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:06.029224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196dba5d-5e73-45ee-aac7-3fccffc7f149 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:06.033149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c40a4-4899-4ed7-b231-be9b7f929e46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:06.034079 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c40a4-4899-4ed7-b231-be9b7f929e46 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:06.034596 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c40a4-4899-4ed7-b231-be9b7f929e46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:06.035135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c40a4-4899-4ed7-b231-be9b7f929e46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:06.045194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c40a4-4899-4ed7-b231-be9b7f929e46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:06.046750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4c40a4-4899-4ed7-b231-be9b7f929e46 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:06.057417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 694/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:07.355879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08f521-bceb-47fd-bf15-3fff252795a6 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:07.356390 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08f521-bceb-47fd-bf15-3fff252795a6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:07.356549 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08f521-bceb-47fd-bf15-3fff252795a6 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:07.358014 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 683/1377] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:07.363305 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58600f1-73ad-4075-9245-c9217ae787c1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:07.363750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58600f1-73ad-4075-9245-c9217ae787c1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:07.363857 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58600f1-73ad-4075-9245-c9217ae787c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:07.381637 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f58600f1-73ad-4075-9245-c9217ae787c1 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:07.382973 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f58600f1-73ad-4075-9245-c9217ae787c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:07.390820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 695/1378] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:10:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:07.440347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08f521-bceb-47fd-bf15-3fff252795a6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:07.455028 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08f521-bceb-47fd-bf15-3fff252795a6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:07.461051 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7871a0-d47f-49de-8285-67ce4166b818 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:07.461884 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7871a0-d47f-49de-8285-67ce4166b818 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:07.461975 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7871a0-d47f-49de-8285-67ce4166b818 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:07.462245 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7871a0-d47f-49de-8285-67ce4166b818 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:07.471041 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7871a0-d47f-49de-8285-67ce4166b818 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:07.472643 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7871a0-d47f-49de-8285-67ce4166b818 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:07.477303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:10:07.483531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 684/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:09.647194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c3695-54ea-4653-9167-926953f47f86 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:09.647718 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c3695-54ea-4653-9167-926953f47f86 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:09.647833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c3695-54ea-4653-9167-926953f47f86 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:09.649942 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 696/1380] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:10:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:09.656328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec558386-31dd-43f6-9130-b5d4620eea4f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:09.657144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec558386-31dd-43f6-9130-b5d4620eea4f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:09.657698 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec558386-31dd-43f6-9130-b5d4620eea4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:09.679509 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ec558386-31dd-43f6-9130-b5d4620eea4f None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:09.680944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ec558386-31dd-43f6-9130-b5d4620eea4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:09.697641 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 685/1381] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:10:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:09.743892 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c3695-54ea-4653-9167-926953f47f86 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:09.760090 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c3695-54ea-4653-9167-926953f47f86 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:09.766905 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb618c0d-52c3-4d0e-ba85-d8fce0929d43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:09.767553 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb618c0d-52c3-4d0e-ba85-d8fce0929d43 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:09.767833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb618c0d-52c3-4d0e-ba85-d8fce0929d43 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:09.770220 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb618c0d-52c3-4d0e-ba85-d8fce0929d43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:09.781244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb618c0d-52c3-4d0e-ba85-d8fce0929d43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:09.783157 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb618c0d-52c3-4d0e-ba85-d8fce0929d43 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:09.793842 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 697/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:11.990916 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bdc73-1332-4e2f-9b35-090d930e8ab5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:11.991843 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bdc73-1332-4e2f-9b35-090d930e8ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:11.992277 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bdc73-1332-4e2f-9b35-090d930e8ab5 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:11.994205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 686/1383] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:10:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:11.999121 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a580ec2-8f8d-4809-915f-9ccc814e348c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:11.999848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a580ec2-8f8d-4809-915f-9ccc814e348c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:12.000030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a580ec2-8f8d-4809-915f-9ccc814e348c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:12.018155 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4a580ec2-8f8d-4809-915f-9ccc814e348c None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:12.019364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4a580ec2-8f8d-4809-915f-9ccc814e348c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:12.028569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 698/1384] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:12.071394 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bdc73-1332-4e2f-9b35-090d930e8ab5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:12.088478 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bdc73-1332-4e2f-9b35-090d930e8ab5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:12.095541 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152146d2-7760-40d1-9029-e583a96e2a09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:12.096189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152146d2-7760-40d1-9029-e583a96e2a09 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:12.096579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152146d2-7760-40d1-9029-e583a96e2a09 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:12.096744 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152146d2-7760-40d1-9029-e583a96e2a09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:12.106903 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152146d2-7760-40d1-9029-e583a96e2a09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:12.108497 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152146d2-7760-40d1-9029-e583a96e2a09 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:12.119802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 687/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:10:14.217083 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc3d0d-bd46-42a1-87af-6d34d24c97f8 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:14.217886 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc3d0d-bd46-42a1-87af-6d34d24c97f8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:14.217886 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc3d0d-bd46-42a1-87af-6d34d24c97f8 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:14.219118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 699/1386] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:14.226323 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ad504-2d68-46f8-95cb-d1e496fc3371 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:14.226871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ad504-2d68-46f8-95cb-d1e496fc3371 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:14.226981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ad504-2d68-46f8-95cb-d1e496fc3371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:14.245810 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c54ad504-2d68-46f8-95cb-d1e496fc3371 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:14.247041 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c54ad504-2d68-46f8-95cb-d1e496fc3371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:14.255259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 688/1387] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:10:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:14.309905 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc3d0d-bd46-42a1-87af-6d34d24c97f8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:14.324861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc3d0d-bd46-42a1-87af-6d34d24c97f8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:14.330495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc5d87-d669-42aa-9c88-93a24673128a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:14.331155 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc5d87-d669-42aa-9c88-93a24673128a service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:14.331413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc5d87-d669-42aa-9c88-93a24673128a service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:14.331696 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc5d87-d669-42aa-9c88-93a24673128a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:14.341146 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc5d87-d669-42aa-9c88-93a24673128a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:14.343336 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc5d87-d669-42aa-9c88-93a24673128a service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:14.357960 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 700/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:16.525918 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b434970-aaa4-4f42-98fc-42952e61aae3 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:16.526524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b434970-aaa4-4f42-98fc-42952e61aae3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:16.526524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b434970-aaa4-4f42-98fc-42952e61aae3 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:16.527893 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 689/1389] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:16.536137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1989dc-5f21-41d4-9177-58a9fef835d7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:16.536599 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1989dc-5f21-41d4-9177-58a9fef835d7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:16.536695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1989dc-5f21-41d4-9177-58a9fef835d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:16.555617 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8c1989dc-5f21-41d4-9177-58a9fef835d7 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:16.556826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8c1989dc-5f21-41d4-9177-58a9fef835d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:16.569781 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 701/1390] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:10:16.614216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b434970-aaa4-4f42-98fc-42952e61aae3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:16.629164 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b434970-aaa4-4f42-98fc-42952e61aae3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:16.635767 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424706ad-e08a-4469-8ded-c791a3540453 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:16.636455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424706ad-e08a-4469-8ded-c791a3540453 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:16.636755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424706ad-e08a-4469-8ded-c791a3540453 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:16.637047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424706ad-e08a-4469-8ded-c791a3540453 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:16.647222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424706ad-e08a-4469-8ded-c791a3540453 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:16.651133 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424706ad-e08a-4469-8ded-c791a3540453 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:16.670127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 690/1391] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:18.937227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eca0ab-8726-46d8-b247-06fc775eeaaf None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:18.937768 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eca0ab-8726-46d8-b247-06fc775eeaaf None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:18.937874 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eca0ab-8726-46d8-b247-06fc775eeaaf None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:18.939268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 702/1392] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:18.944596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95961e6d-9850-4b2f-98aa-baefe1059aa8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:18.945118 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95961e6d-9850-4b2f-98aa-baefe1059aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:18.945214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95961e6d-9850-4b2f-98aa-baefe1059aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:18.963596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-95961e6d-9850-4b2f-98aa-baefe1059aa8 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:18.964803 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-95961e6d-9850-4b2f-98aa-baefe1059aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:18.976761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 691/1393] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:19.050617 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eca0ab-8726-46d8-b247-06fc775eeaaf None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:19.075142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eca0ab-8726-46d8-b247-06fc775eeaaf None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:19.084487 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a1e5f3-fc77-44e2-ba55-9a8795be1ec8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:19.085677 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1e5f3-fc77-44e2-ba55-9a8795be1ec8 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:19.086474 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1e5f3-fc77-44e2-ba55-9a8795be1ec8 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:19.087046 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a1e5f3-fc77-44e2-ba55-9a8795be1ec8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:19.103154 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a1e5f3-fc77-44e2-ba55-9a8795be1ec8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:19.106258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a1e5f3-fc77-44e2-ba55-9a8795be1ec8 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:19.125081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 703/1394] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:21.306708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991ad8b-0f82-4130-9f61-c574dbc4c579 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:21.307559 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991ad8b-0f82-4130-9f61-c574dbc4c579 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:21.307849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991ad8b-0f82-4130-9f61-c574dbc4c579 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:21.309372 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 692/1395] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:10:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:21.314757 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d861a-1294-473c-9edc-76437e3b22fb None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:21.315157 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d861a-1294-473c-9edc-76437e3b22fb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:21.315325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d861a-1294-473c-9edc-76437e3b22fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:21.333515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-279d861a-1294-473c-9edc-76437e3b22fb None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:21.334750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-279d861a-1294-473c-9edc-76437e3b22fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:21.343287 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 704/1396] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:10:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:21.387928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991ad8b-0f82-4130-9f61-c574dbc4c579 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:21.403069 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991ad8b-0f82-4130-9f61-c574dbc4c579 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:21.408649 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7910012-9171-44d9-9c7d-a068e381d08a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:21.409299 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7910012-9171-44d9-9c7d-a068e381d08a service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:21.409587 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7910012-9171-44d9-9c7d-a068e381d08a service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:21.409870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7910012-9171-44d9-9c7d-a068e381d08a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:21.418566 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7910012-9171-44d9-9c7d-a068e381d08a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:21.420156 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7910012-9171-44d9-9c7d-a068e381d08a service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:21.429587 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 693/1397] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:21.658970 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0718066-9b71-4499-943d-80256d8414c0 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:21.659263 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0718066-9b71-4499-943d-80256d8414c0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:21.659547 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0718066-9b71-4499-943d-80256d8414c0 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:21.660972 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 705/1398] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:21.666995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c8aab-ad3c-4771-b11e-28d55d267287 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:21.667295 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c8aab-ad3c-4771-b11e-28d55d267287 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:21.667582 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c8aab-ad3c-4771-b11e-28d55d267287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:21.686943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-420c8aab-ad3c-4771-b11e-28d55d267287 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:21.688184 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-420c8aab-ad3c-4771-b11e-28d55d267287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:21.696927 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 694/1399] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:10:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:21.754790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c2529-564b-45d5-9012-55e48a2a29f0 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:21.755083 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c2529-564b-45d5-9012-55e48a2a29f0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:21.755367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c2529-564b-45d5-9012-55e48a2a29f0 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:21.756776 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 706/1400] 10.222.0.16 () {54 vars in 962 bytes} [Fri Jun 24 04:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:21.762823 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936849e-890b-43a3-b7d5-0cc3baed41f8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:21.763117 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936849e-890b-43a3-b7d5-0cc3baed41f8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:21.763404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936849e-890b-43a3-b7d5-0cc3baed41f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:21.812135 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b936849e-890b-43a3-b7d5-0cc3baed41f8 None None] MFA Rules not processed for user `61459e15d327405b88a2fdfc1fc0fecd`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:21.813371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b936849e-890b-43a3-b7d5-0cc3baed41f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:21.873017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 695/1401] 10.222.0.16 () {58 vars in 1059 bytes} [Fri Jun 24 04:10:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:21.880925 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c2529-564b-45d5-9012-55e48a2a29f0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:21.892026 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c2529-564b-45d5-9012-55e48a2a29f0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:21.895548 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ec1a6-beed-4bf7-8d4d-29a2750b2348 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61459e15d327405b88a2fdfc1fc0fecd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:21.896186 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ec1a6-beed-4bf7-8d4d-29a2750b2348 service glance] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:21.896471 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ec1a6-beed-4bf7-8d4d-29a2750b2348 service glance] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:21.896752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ec1a6-beed-4bf7-8d4d-29a2750b2348 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:21.906677 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ec1a6-beed-4bf7-8d4d-29a2750b2348 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:21.908125 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ec1a6-beed-4bf7-8d4d-29a2750b2348 service glance] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:21.917706 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 707/1402] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:22.740123 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1330f-9557-4774-a4f8-b97f84d31c9c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:22.740577 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1330f-9557-4774-a4f8-b97f84d31c9c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:22.740690 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1330f-9557-4774-a4f8-b97f84d31c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:22.791466 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-69b1330f-9557-4774-a4f8-b97f84d31c9c None None] MFA Rules not processed for user `98f58f91d17949198e66883d234a9e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:22.792653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-69b1330f-9557-4774-a4f8-b97f84d31c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:22.832772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:10:22.857387 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 696/1403] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jun 24 04:10:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:22.902697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6ebb0-a7b3-412d-87a6-2190cc60916f None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:22.903173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6ebb0-a7b3-412d-87a6-2190cc60916f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:22.903272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6ebb0-a7b3-412d-87a6-2190cc60916f None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:22.904608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 708/1404] 10.222.0.16 () {54 vars in 942 bytes} [Fri Jun 24 04:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:22.911158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5dd88b-d166-45a6-ab48-1396735e95a7 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:22.911448 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5dd88b-d166-45a6-ab48-1396735e95a7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:22.911737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5dd88b-d166-45a6-ab48-1396735e95a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:22.959675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-5b5dd88b-d166-45a6-ab48-1396735e95a7 None None] MFA Rules not processed for user `6d8e1f1e3c7e42cc9fa6f82aaac55129`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:22.961487 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5b5dd88b-d166-45a6-ab48-1396735e95a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:23.036140 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 697/1405] 10.222.0.16 () {58 vars in 1039 bytes} [Fri Jun 24 04:10:22 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:23.045843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6ebb0-a7b3-412d-87a6-2190cc60916f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:23.053321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=61 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:10:23.060360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6ebb0-a7b3-412d-87a6-2190cc60916f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:23.065366 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617649db-096b-45b6-992b-a01c0da07696 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8e1f1e3c7e42cc9fa6f82aaac55129', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:23.066170 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617649db-096b-45b6-992b-a01c0da07696 service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:23.066494 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617649db-096b-45b6-992b-a01c0da07696 service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:23.066776 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617649db-096b-45b6-992b-a01c0da07696 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:23.075723 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617649db-096b-45b6-992b-a01c0da07696 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:23.077301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617649db-096b-45b6-992b-a01c0da07696 service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:23.086139 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 709/1406] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jun 24 04:10:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 04:10:23.640508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097d95f-c6d9-4eac-8078-09079d6b4494 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:23.640800 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097d95f-c6d9-4eac-8078-09079d6b4494 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:23.641084 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097d95f-c6d9-4eac-8078-09079d6b4494 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:23.642666 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 698/1407] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:23.647865 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a0516-3e95-49ab-adf3-500c58811a3d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:23.648243 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a0516-3e95-49ab-adf3-500c58811a3d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:23.648542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a0516-3e95-49ab-adf3-500c58811a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:23.677722 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-940a0516-3e95-49ab-adf3-500c58811a3d None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:23.679753 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-940a0516-3e95-49ab-adf3-500c58811a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:23.690416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 710/1408] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:10:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:23.733835 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097d95f-c6d9-4eac-8078-09079d6b4494 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:23.750133 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097d95f-c6d9-4eac-8078-09079d6b4494 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:23.755740 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad868d6-8678-424f-8bdc-48ef2f5ca21b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:23.756379 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad868d6-8678-424f-8bdc-48ef2f5ca21b service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:23.756663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad868d6-8678-424f-8bdc-48ef2f5ca21b service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:23.756996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad868d6-8678-424f-8bdc-48ef2f5ca21b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:23.765730 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad868d6-8678-424f-8bdc-48ef2f5ca21b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:23.767345 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad868d6-8678-424f-8bdc-48ef2f5ca21b service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:23.778894 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 699/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:33.661788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=8 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:10:33.773513 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=59 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:10:46.926268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d169602-9ba6-4723-a9c0-c5f5451b86b3 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:46.926877 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d169602-9ba6-4723-a9c0-c5f5451b86b3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:46.926877 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d169602-9ba6-4723-a9c0-c5f5451b86b3 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:46.928250 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 711/1410] 10.222.0.16 () {54 vars in 918 bytes} [Fri Jun 24 04:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:46.934060 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ba433-ac06-4b71-8074-78046cdd82aa None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:46.934514 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ba433-ac06-4b71-8074-78046cdd82aa None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:46.934647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ba433-ac06-4b71-8074-78046cdd82aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:46.952678 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c15ba433-ac06-4b71-8074-78046cdd82aa None None] MFA Rules not processed for user `7662e61619d44c92af3ffacbacba3913`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:46.953867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c15ba433-ac06-4b71-8074-78046cdd82aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:46.962893 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 700/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Fri Jun 24 04:10:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:46.977684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13324f-6f72-43ff-9595-7d0d44224e54 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:46.977971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13324f-6f72-43ff-9595-7d0d44224e54 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:46.978248 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13324f-6f72-43ff-9595-7d0d44224e54 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:46.979618 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 712/1412] 10.222.0.16 () {54 vars in 918 bytes} [Fri Jun 24 04:10:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:46.984097 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e4fe8-dfcd-425c-8ba9-6a1976b546a4 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:46.984383 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e4fe8-dfcd-425c-8ba9-6a1976b546a4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:46.984689 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e4fe8-dfcd-425c-8ba9-6a1976b546a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:47.002015 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-701e4fe8-dfcd-425c-8ba9-6a1976b546a4 None None] MFA Rules not processed for user `80b7b3ec60db4de19bbf7d08f111f470`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:47.003234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-701e4fe8-dfcd-425c-8ba9-6a1976b546a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:47.012129 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 701/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Fri Jun 24 04:10:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:49.528638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8604ac-5dcc-4242-ae94-474307824a33 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:49.529205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8604ac-5dcc-4242-ae94-474307824a33 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:49.529205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8604ac-5dcc-4242-ae94-474307824a33 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:49.530627 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 04:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:49.535841 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a884645-fa8e-4ff5-889b-d00a7f099111 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:49.536294 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a884645-fa8e-4ff5-889b-d00a7f099111 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:49.536399 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a884645-fa8e-4ff5-889b-d00a7f099111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:49.553896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7a884645-fa8e-4ff5-889b-d00a7f099111 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:49.555799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7a884645-fa8e-4ff5-889b-d00a7f099111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:49.563618 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 04:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:49.612958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8604ac-5dcc-4242-ae94-474307824a33 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:49.628577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8604ac-5dcc-4242-ae94-474307824a33 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:49.634411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1507f-1c67-4aeb-8861-b4968d0e6e9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:49.635049 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1507f-1c67-4aeb-8861-b4968d0e6e9b service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:49.635333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1507f-1c67-4aeb-8861-b4968d0e6e9b service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:49.635616 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1507f-1c67-4aeb-8861-b4968d0e6e9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:49.643849 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1507f-1c67-4aeb-8861-b4968d0e6e9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:49.646287 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1507f-1c67-4aeb-8861-b4968d0e6e9b service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:49.655027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 714/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:50.553422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ea550-785d-4ea5-a5bb-55221ddb03d6 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:50.554015 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ea550-785d-4ea5-a5bb-55221ddb03d6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:50.554015 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ea550-785d-4ea5-a5bb-55221ddb03d6 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:50.555542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 04:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:50.560418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655500d-951f-476b-90fc-c033b02bc944 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:50.560820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655500d-951f-476b-90fc-c033b02bc944 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:50.560974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655500d-951f-476b-90fc-c033b02bc944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:50.579115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b655500d-951f-476b-90fc-c033b02bc944 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:50.580300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b655500d-951f-476b-90fc-c033b02bc944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:50.588580 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 04:10:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:50.740259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ea550-785d-4ea5-a5bb-55221ddb03d6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:50.754706 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ea550-785d-4ea5-a5bb-55221ddb03d6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:50.760180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5ec80-e092-4eec-bbc4-441c89fa642c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:50.760809 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5ec80-e092-4eec-bbc4-441c89fa642c service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:50.761102 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5ec80-e092-4eec-bbc4-441c89fa642c service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:50.761373 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5ec80-e092-4eec-bbc4-441c89fa642c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:50.769366 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5ec80-e092-4eec-bbc4-441c89fa642c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:50.771009 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5ec80-e092-4eec-bbc4-441c89fa642c service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:50.781056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 704/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:51.677945 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533c026-31e0-4655-a4d4-8ca1d0baff90 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:51.678610 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533c026-31e0-4655-a4d4-8ca1d0baff90 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:51.678610 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533c026-31e0-4655-a4d4-8ca1d0baff90 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:51.680030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 04:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:51.685068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaa832-2f23-45c1-8e10-3a5dd4e7e68e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:51.685524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaa832-2f23-45c1-8e10-3a5dd4e7e68e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:51.685637 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaa832-2f23-45c1-8e10-3a5dd4e7e68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:51.704015 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2edaa832-2f23-45c1-8e10-3a5dd4e7e68e None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:51.705207 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2edaa832-2f23-45c1-8e10-3a5dd4e7e68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:51.713649 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 04:10:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:51.864262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533c026-31e0-4655-a4d4-8ca1d0baff90 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:51.879093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533c026-31e0-4655-a4d4-8ca1d0baff90 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:51.884675 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c82fd3-65b1-4698-85b8-0a5066d272af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:51.885307 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c82fd3-65b1-4698-85b8-0a5066d272af service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:51.893305 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c82fd3-65b1-4698-85b8-0a5066d272af service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:51.893305 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c82fd3-65b1-4698-85b8-0a5066d272af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:51.893814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c82fd3-65b1-4698-85b8-0a5066d272af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:51.895424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c82fd3-65b1-4698-85b8-0a5066d272af service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:51.905046 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 717/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:52.783143 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb533b70-7541-46c6-961d-45b283368098 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:52.783723 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb533b70-7541-46c6-961d-45b283368098 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:52.783723 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb533b70-7541-46c6-961d-45b283368098 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:52.785066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 04:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:52.790452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a7bd7-baa3-4937-85cd-24472544c82e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:52.790889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a7bd7-baa3-4937-85cd-24472544c82e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:52.791001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a7bd7-baa3-4937-85cd-24472544c82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:52.809014 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-425a7bd7-baa3-4937-85cd-24472544c82e None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:52.810194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-425a7bd7-baa3-4937-85cd-24472544c82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:52.818205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 04:10:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:52.966602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb533b70-7541-46c6-961d-45b283368098 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:52.981600 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb533b70-7541-46c6-961d-45b283368098 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:52.986984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d64d59-44a6-409e-98f6-feea605200a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:52.987623 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d64d59-44a6-409e-98f6-feea605200a8 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:52.987902 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d64d59-44a6-409e-98f6-feea605200a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:52.988178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d64d59-44a6-409e-98f6-feea605200a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:52.996481 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d64d59-44a6-409e-98f6-feea605200a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:52.998063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d64d59-44a6-409e-98f6-feea605200a8 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:53.007691 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 707/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:53.880437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76b7c4-3830-455c-8d04-4b5b998338b2 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:53.881021 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76b7c4-3830-455c-8d04-4b5b998338b2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:53.881021 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f76b7c4-3830-455c-8d04-4b5b998338b2 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:53.882560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 04:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:53.887558 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db90ca-221d-40f3-bf6c-d857bfa902e6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:53.887994 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db90ca-221d-40f3-bf6c-d857bfa902e6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:53.888109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db90ca-221d-40f3-bf6c-d857bfa902e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:53.906113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-05db90ca-221d-40f3-bf6c-d857bfa902e6 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:53.907339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-05db90ca-221d-40f3-bf6c-d857bfa902e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:53.915721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 04:10:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:54.063473 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76b7c4-3830-455c-8d04-4b5b998338b2 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:54.078211 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f76b7c4-3830-455c-8d04-4b5b998338b2 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:54.083770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b664ca-e93a-40e6-8f50-f3621deafa2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:54.084422 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b664ca-e93a-40e6-8f50-f3621deafa2d service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:54.084704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b664ca-e93a-40e6-8f50-f3621deafa2d service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:54.084990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b664ca-e93a-40e6-8f50-f3621deafa2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:54.093778 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b664ca-e93a-40e6-8f50-f3621deafa2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:54.095416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b664ca-e93a-40e6-8f50-f3621deafa2d service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:54.104774 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 720/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:55.002471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb5558-5e14-4f05-b5f9-2c163f92cdd1 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:55.002991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb5558-5e14-4f05-b5f9-2c163f92cdd1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:55.003101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb5558-5e14-4f05-b5f9-2c163f92cdd1 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:55.004407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 04:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:55.009863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa79f1-7987-4a69-896d-6480122ff56d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:55.010316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa79f1-7987-4a69-896d-6480122ff56d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:55.010440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caa79f1-7987-4a69-896d-6480122ff56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:55.028705 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1caa79f1-7987-4a69-896d-6480122ff56d None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:55.029949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1caa79f1-7987-4a69-896d-6480122ff56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:55.037847 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 04:10:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:55.190209 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb5558-5e14-4f05-b5f9-2c163f92cdd1 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:55.204886 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb5558-5e14-4f05-b5f9-2c163f92cdd1 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:55.210653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0467802-0d47-47f2-a7f8-c764510dfa9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:55.211286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0467802-0d47-47f2-a7f8-c764510dfa9a service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:55.211568 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0467802-0d47-47f2-a7f8-c764510dfa9a service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:55.211848 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0467802-0d47-47f2-a7f8-c764510dfa9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:55.220298 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0467802-0d47-47f2-a7f8-c764510dfa9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:55.221869 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0467802-0d47-47f2-a7f8-c764510dfa9a service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:55.231813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 710/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:56.104944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f9a1b-3dd1-49cc-bc70-447aa9d0f1c5 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:56.105451 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f9a1b-3dd1-49cc-bc70-447aa9d0f1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:56.105565 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f9a1b-3dd1-49cc-bc70-447aa9d0f1c5 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:56.106929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 04:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:56.111861 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f79fd-5c11-4a0b-a412-104a3df9f2d5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:56.112311 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f79fd-5c11-4a0b-a412-104a3df9f2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:56.112423 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f79fd-5c11-4a0b-a412-104a3df9f2d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:56.130388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c13f79fd-5c11-4a0b-a412-104a3df9f2d5 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:56.131626 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c13f79fd-5c11-4a0b-a412-104a3df9f2d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:56.139725 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 04:10:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:56.291628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f9a1b-3dd1-49cc-bc70-447aa9d0f1c5 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:56.305915 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f9a1b-3dd1-49cc-bc70-447aa9d0f1c5 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:56.311208 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da04fa5-737e-40ec-93fc-a4a1b16afa09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:56.311840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da04fa5-737e-40ec-93fc-a4a1b16afa09 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:56.312125 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da04fa5-737e-40ec-93fc-a4a1b16afa09 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:56.312403 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da04fa5-737e-40ec-93fc-a4a1b16afa09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:56.321050 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da04fa5-737e-40ec-93fc-a4a1b16afa09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:56.322676 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da04fa5-737e-40ec-93fc-a4a1b16afa09 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:56.331939 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 723/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:57.205352 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ade3c-34cd-4747-b17f-ddf84fc7daba None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:57.205920 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ade3c-34cd-4747-b17f-ddf84fc7daba None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:57.205920 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ade3c-34cd-4747-b17f-ddf84fc7daba None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:57.207250 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 04:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:57.212496 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1796fd-8ba5-4ffc-8165-f929359a0143 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:57.212914 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1796fd-8ba5-4ffc-8165-f929359a0143 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:57.213054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1796fd-8ba5-4ffc-8165-f929359a0143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:57.231517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-cc1796fd-8ba5-4ffc-8165-f929359a0143 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:57.232721 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-cc1796fd-8ba5-4ffc-8165-f929359a0143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:57.240697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 04:10:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:57.391639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ade3c-34cd-4747-b17f-ddf84fc7daba None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:57.405883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ade3c-34cd-4747-b17f-ddf84fc7daba None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:57.411219 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc2398-4143-4d0a-89e6-d7c4554081ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:57.411848 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc2398-4143-4d0a-89e6-d7c4554081ce service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:57.412133 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc2398-4143-4d0a-89e6-d7c4554081ce service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:57.412411 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc2398-4143-4d0a-89e6-d7c4554081ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:57.420951 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc2398-4143-4d0a-89e6-d7c4554081ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:57.422563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc2398-4143-4d0a-89e6-d7c4554081ce service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:57.432962 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 713/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:10:58.295481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b1df4-5b63-49f9-aa14-1b60783cec09 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:58.295997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b1df4-5b63-49f9-aa14-1b60783cec09 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:58.295997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b1df4-5b63-49f9-aa14-1b60783cec09 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:58.297364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 04:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:10:58.302466 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2113837-1a96-42ed-8624-68c640103352 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:58.302896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2113837-1a96-42ed-8624-68c640103352 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:58.303002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2113837-1a96-42ed-8624-68c640103352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:58.323134 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c2113837-1a96-42ed-8624-68c640103352 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:10:58.324324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c2113837-1a96-42ed-8624-68c640103352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:10:58.331965 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 04:10:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:10:58.481487 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b1df4-5b63-49f9-aa14-1b60783cec09 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:10:58.496676 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b1df4-5b63-49f9-aa14-1b60783cec09 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:10:58.502175 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca1c7f-2617-47bb-b856-f12eab221ecc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:10:58.502837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca1c7f-2617-47bb-b856-f12eab221ecc service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:10:58.503124 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca1c7f-2617-47bb-b856-f12eab221ecc service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:10:58.503402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca1c7f-2617-47bb-b856-f12eab221ecc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:10:58.512011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca1c7f-2617-47bb-b856-f12eab221ecc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:10:58.513579 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca1c7f-2617-47bb-b856-f12eab221ecc service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:10:58.522825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 726/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:10:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:11:00.553130 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407ba90-23f3-4ca3-b637-5a45e8682515 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:00.553699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407ba90-23f3-4ca3-b637-5a45e8682515 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:00.553699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407ba90-23f3-4ca3-b637-5a45e8682515 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:00.555652 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 04:11:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:11:00.560991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b6ea7-9aa0-4963-a361-4d78c2df5ffd None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:00.562193 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b6ea7-9aa0-4963-a361-4d78c2df5ffd None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:00.562193 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b6ea7-9aa0-4963-a361-4d78c2df5ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:00.580043 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-a75b6ea7-9aa0-4963-a361-4d78c2df5ffd None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:11:00.581241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a75b6ea7-9aa0-4963-a361-4d78c2df5ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:11:00.589118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 04:11:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:11:00.627750 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407ba90-23f3-4ca3-b637-5a45e8682515 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:11:00.638521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407ba90-23f3-4ca3-b637-5a45e8682515 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:11:00.642005 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545ef28-4441-4d11-ab5d-6e3c31772f78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:11:00.642678 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545ef28-4441-4d11-ab5d-6e3c31772f78 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:00.642968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545ef28-4441-4d11-ab5d-6e3c31772f78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:00.643251 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545ef28-4441-4d11-ab5d-6e3c31772f78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:00.652514 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545ef28-4441-4d11-ab5d-6e3c31772f78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:11:00.653946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545ef28-4441-4d11-ab5d-6e3c31772f78 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:11:00.663799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 716/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:11:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:11:02.834504 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf8fd7-100f-4e73-9f9e-66a25f4f4cb8 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:02.835579 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf8fd7-100f-4e73-9f9e-66a25f4f4cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:02.835704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf8fd7-100f-4e73-9f9e-66a25f4f4cb8 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:02.836983 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 04:11:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:11:02.842221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1592b17-1ec7-4dde-b1a4-de4bb7e4a9b1 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:02.842703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1592b17-1ec7-4dde-b1a4-de4bb7e4a9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:02.842879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1592b17-1ec7-4dde-b1a4-de4bb7e4a9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:02.861270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a1592b17-1ec7-4dde-b1a4-de4bb7e4a9b1 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:11:02.862516 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a1592b17-1ec7-4dde-b1a4-de4bb7e4a9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:11:02.870500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 04:11:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:11:02.908281 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf8fd7-100f-4e73-9f9e-66a25f4f4cb8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:11:02.919501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf8fd7-100f-4e73-9f9e-66a25f4f4cb8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:11:02.922987 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45f863-cc0e-4d9a-b84d-1c4b53a3b74a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:11:02.923619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45f863-cc0e-4d9a-b84d-1c4b53a3b74a service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:02.923908 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45f863-cc0e-4d9a-b84d-1c4b53a3b74a service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:02.924188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45f863-cc0e-4d9a-b84d-1c4b53a3b74a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:02.933189 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45f863-cc0e-4d9a-b84d-1c4b53a3b74a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:11:02.934664 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45f863-cc0e-4d9a-b84d-1c4b53a3b74a service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:11:02.944085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 729/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:11:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:11:05.401850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d07fe-f08a-4ac8-9ade-c45c017beeea None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:05.402405 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d07fe-f08a-4ac8-9ade-c45c017beeea None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:05.402504 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d07fe-f08a-4ac8-9ade-c45c017beeea None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:05.403850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 04:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:11:05.409135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311eab96-c8d7-4f19-887f-5a7ed0cbc844 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:05.409571 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311eab96-c8d7-4f19-887f-5a7ed0cbc844 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:05.409684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311eab96-c8d7-4f19-887f-5a7ed0cbc844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:05.428699 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-311eab96-c8d7-4f19-887f-5a7ed0cbc844 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:11:05.429995 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-311eab96-c8d7-4f19-887f-5a7ed0cbc844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:11:05.438328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 04:11:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:11:05.479095 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d07fe-f08a-4ac8-9ade-c45c017beeea None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:11:05.489704 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d07fe-f08a-4ac8-9ade-c45c017beeea None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:11:05.493970 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e679469-69f1-4e48-8a1d-edf173d94043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:11:05.494644 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e679469-69f1-4e48-8a1d-edf173d94043 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:05.494929 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e679469-69f1-4e48-8a1d-edf173d94043 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:05.495206 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e679469-69f1-4e48-8a1d-edf173d94043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:05.504310 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e679469-69f1-4e48-8a1d-edf173d94043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:11:05.505737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e679469-69f1-4e48-8a1d-edf173d94043 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:11:05.515335 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 719/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:11:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:11:07.702927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c0d6a-45bd-4566-99f3-4ef58ba8d65e None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:07.703531 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c0d6a-45bd-4566-99f3-4ef58ba8d65e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:07.703531 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c0d6a-45bd-4566-99f3-4ef58ba8d65e None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:07.704862 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 04:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:11:07.710173 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d02b3-80f6-4bdf-ada1-94479e9f4728 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:07.710610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d02b3-80f6-4bdf-ada1-94479e9f4728 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:07.710834 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d02b3-80f6-4bdf-ada1-94479e9f4728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:07.729675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-267d02b3-80f6-4bdf-ada1-94479e9f4728 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:11:07.730895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-267d02b3-80f6-4bdf-ada1-94479e9f4728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:11:07.738470 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 04:11:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:11:07.778315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c0d6a-45bd-4566-99f3-4ef58ba8d65e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:11:07.789047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c0d6a-45bd-4566-99f3-4ef58ba8d65e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:11:07.792790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9984ad-f1a5-4b11-9004-878940342937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:11:07.793481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9984ad-f1a5-4b11-9004-878940342937 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:07.793787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9984ad-f1a5-4b11-9004-878940342937 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:07.794063 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9984ad-f1a5-4b11-9004-878940342937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:07.803043 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9984ad-f1a5-4b11-9004-878940342937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:11:07.804470 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9984ad-f1a5-4b11-9004-878940342937 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:11:07.813852 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 732/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:11:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:11:09.904433 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8563f1-0c98-4b69-8e36-d4d3d4a74ac2 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:09.904998 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8563f1-0c98-4b69-8e36-d4d3d4a74ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:09.904998 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8563f1-0c98-4b69-8e36-d4d3d4a74ac2 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:09.906457 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 04:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:11:09.911818 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23123d-3fa9-4683-bf66-7a8d8bae2d4f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:09.912245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23123d-3fa9-4683-bf66-7a8d8bae2d4f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:09.912401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23123d-3fa9-4683-bf66-7a8d8bae2d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:09.930633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-de23123d-3fa9-4683-bf66-7a8d8bae2d4f None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:11:09.931828 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-de23123d-3fa9-4683-bf66-7a8d8bae2d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:11:09.939502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 04:11:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:11:09.976398 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8563f1-0c98-4b69-8e36-d4d3d4a74ac2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:11:09.987480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8563f1-0c98-4b69-8e36-d4d3d4a74ac2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:11:09.990932 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717bdf00-633a-4ebe-ab3c-d362786f50ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:11:09.991561 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bdf00-633a-4ebe-ab3c-d362786f50ea service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:09.991846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bdf00-633a-4ebe-ab3c-d362786f50ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:09.992125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bdf00-633a-4ebe-ab3c-d362786f50ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:10.001782 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717bdf00-633a-4ebe-ab3c-d362786f50ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:11:10.003244 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717bdf00-633a-4ebe-ab3c-d362786f50ea service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:11:10.012646 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 722/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:11:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:11:12.117714 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e035eac-079d-48a0-a541-1894a49000b6 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:12.118283 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e035eac-079d-48a0-a541-1894a49000b6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:12.118283 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e035eac-079d-48a0-a541-1894a49000b6 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:12.119740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 04:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:11:12.124855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4b4c1-f98b-4b92-8398-50d2c99981e4 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:12.125501 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4b4c1-f98b-4b92-8398-50d2c99981e4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:12.125679 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4b4c1-f98b-4b92-8398-50d2c99981e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:12.143460 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3cd4b4c1-f98b-4b92-8398-50d2c99981e4 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:11:12.144643 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3cd4b4c1-f98b-4b92-8398-50d2c99981e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:11:12.152892 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 04:11:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:11:12.189757 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e035eac-079d-48a0-a541-1894a49000b6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:11:12.201887 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e035eac-079d-48a0-a541-1894a49000b6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:11:12.205378 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0e10f-66c1-4c4d-9674-13001a259b82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:11:12.206009 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0e10f-66c1-4c4d-9674-13001a259b82 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:12.206301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0e10f-66c1-4c4d-9674-13001a259b82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:12.206610 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0e10f-66c1-4c4d-9674-13001a259b82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:12.215565 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0e10f-66c1-4c4d-9674-13001a259b82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:11:12.216990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d0e10f-66c1-4c4d-9674-13001a259b82 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:11:12.226992 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 735/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:11:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:11:14.372655 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c718424-5af8-4fb0-b5e2-c9be52bb8303 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:14.373196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c718424-5af8-4fb0-b5e2-c9be52bb8303 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:14.373196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c718424-5af8-4fb0-b5e2-c9be52bb8303 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:14.374671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 04:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:11:14.380213 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b3049-eb18-42a6-9be1-03e4af32f058 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:14.380660 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b3049-eb18-42a6-9be1-03e4af32f058 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:14.380772 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b3049-eb18-42a6-9be1-03e4af32f058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:14.387796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=70 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:11:14.400522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-375b3049-eb18-42a6-9be1-03e4af32f058 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:11:14.401711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-375b3049-eb18-42a6-9be1-03e4af32f058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:11:14.409334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 04:11:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:11:14.447321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c718424-5af8-4fb0-b5e2-c9be52bb8303 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:11:14.452694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=73 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:11:14.458477 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c718424-5af8-4fb0-b5e2-c9be52bb8303 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:11:14.462652 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851cce30-6ead-4acc-94df-b5d0a4813a3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:11:14.463285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cce30-6ead-4acc-94df-b5d0a4813a3c service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:14.463570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cce30-6ead-4acc-94df-b5d0a4813a3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:14.463847 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851cce30-6ead-4acc-94df-b5d0a4813a3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:14.473146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851cce30-6ead-4acc-94df-b5d0a4813a3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:11:14.474626 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851cce30-6ead-4acc-94df-b5d0a4813a3c service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:11:14.484513 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 725/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:11:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:11:16.579443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a95155-e5f4-4aed-a9b5-48ffbaa618e3 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:16.580014 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a95155-e5f4-4aed-a9b5-48ffbaa618e3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:16.580014 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a95155-e5f4-4aed-a9b5-48ffbaa618e3 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:16.581426 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 04:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:11:16.586660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74d79c-5665-4c67-9569-172362f1e422 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:16.587065 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74d79c-5665-4c67-9569-172362f1e422 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:16.587217 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74d79c-5665-4c67-9569-172362f1e422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:16.605468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-5c74d79c-5665-4c67-9569-172362f1e422 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:11:16.606679 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5c74d79c-5665-4c67-9569-172362f1e422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:11:16.614280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 04:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:11:16.653039 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a95155-e5f4-4aed-a9b5-48ffbaa618e3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:11:16.663517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a95155-e5f4-4aed-a9b5-48ffbaa618e3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:11:16.667117 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6f39f-06c6-42f6-be75-15dcf1cba224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:11:16.667734 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6f39f-06c6-42f6-be75-15dcf1cba224 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:16.668018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6f39f-06c6-42f6-be75-15dcf1cba224 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:16.668295 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6f39f-06c6-42f6-be75-15dcf1cba224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:16.677017 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6f39f-06c6-42f6-be75-15dcf1cba224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:11:16.678486 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6f39f-06c6-42f6-be75-15dcf1cba224 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:11:16.687766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 738/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:11:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:11:18.725358 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075f24e-369f-40b8-82fc-8fe7f3e2461c None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:18.725945 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075f24e-369f-40b8-82fc-8fe7f3e2461c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:18.725945 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4075f24e-369f-40b8-82fc-8fe7f3e2461c None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:18.727961 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 04:11:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:11:18.733143 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d12b5-8779-4795-9a2c-3ceb2be0ed6e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:18.733534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d12b5-8779-4795-9a2c-3ceb2be0ed6e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:18.733713 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39d12b5-8779-4795-9a2c-3ceb2be0ed6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:18.751381 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c39d12b5-8779-4795-9a2c-3ceb2be0ed6e None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:11:18.752575 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c39d12b5-8779-4795-9a2c-3ceb2be0ed6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:11:18.760275 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 04:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:11:18.797602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075f24e-369f-40b8-82fc-8fe7f3e2461c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:11:18.809997 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4075f24e-369f-40b8-82fc-8fe7f3e2461c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:11:18.813482 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012804d7-3260-4649-915d-71bd5108cde8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:11:18.814116 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012804d7-3260-4649-915d-71bd5108cde8 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:18.814425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012804d7-3260-4649-915d-71bd5108cde8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:18.814705 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012804d7-3260-4649-915d-71bd5108cde8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:18.824678 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012804d7-3260-4649-915d-71bd5108cde8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:11:18.826100 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012804d7-3260-4649-915d-71bd5108cde8 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:11:18.835411 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 728/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:11:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:11:20.928578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a69ca-e708-489c-9b59-65f02c478194 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:20.929159 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a69ca-e708-489c-9b59-65f02c478194 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:20.929159 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a69ca-e708-489c-9b59-65f02c478194 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:20.930558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 04:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:11:20.935743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94643484-b5d4-4e14-804b-d77edadcdaf7 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:20.936151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94643484-b5d4-4e14-804b-d77edadcdaf7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:20.936306 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94643484-b5d4-4e14-804b-d77edadcdaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:20.954059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-94643484-b5d4-4e14-804b-d77edadcdaf7 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:11:20.955289 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-94643484-b5d4-4e14-804b-d77edadcdaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:11:20.963859 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 04:11:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:11:21.000596 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a69ca-e708-489c-9b59-65f02c478194 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:11:21.011107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a69ca-e708-489c-9b59-65f02c478194 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:11:21.014506 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ba8e1-521f-4518-afaf-15843d9fb5fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:11:21.015134 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ba8e1-521f-4518-afaf-15843d9fb5fe service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:21.015517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ba8e1-521f-4518-afaf-15843d9fb5fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:21.015798 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ba8e1-521f-4518-afaf-15843d9fb5fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:21.024971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ba8e1-521f-4518-afaf-15843d9fb5fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:11:21.026406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ba8e1-521f-4518-afaf-15843d9fb5fe service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:11:21.035872 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 741/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:11:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:11:30.949009 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=17 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:11:31.031592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=17 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:11:42.099934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03dd9aa-5b72-4d6c-be36-1b15236fb588 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:42.100527 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03dd9aa-5b72-4d6c-be36-1b15236fb588 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:42.100527 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03dd9aa-5b72-4d6c-be36-1b15236fb588 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:42.101923 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 730/1471] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:11:42.107363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96c8e6-a0e4-46a4-8369-465adbc90230 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:42.107828 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96c8e6-a0e4-46a4-8369-465adbc90230 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:42.107927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96c8e6-a0e4-46a4-8369-465adbc90230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:42.125658 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-eb96c8e6-a0e4-46a4-8369-465adbc90230 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:11:42.126881 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-eb96c8e6-a0e4-46a4-8369-465adbc90230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:11:42.135186 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 742/1472] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:11:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:11:42.183046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc37d31-3a9e-4f56-be68-2c35796f59e6 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:42.183338 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc37d31-3a9e-4f56-be68-2c35796f59e6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:42.183620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc37d31-3a9e-4f56-be68-2c35796f59e6 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:42.184993 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 731/1473] 10.222.0.16 () {54 vars in 962 bytes} [Fri Jun 24 04:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:11:42.189764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac13ed-85c2-4a8e-974e-d1d28eb19012 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:42.190055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac13ed-85c2-4a8e-974e-d1d28eb19012 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:42.190360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac13ed-85c2-4a8e-974e-d1d28eb19012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:42.208873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d9ac13ed-85c2-4a8e-974e-d1d28eb19012 None None] MFA Rules not processed for user `61459e15d327405b88a2fdfc1fc0fecd`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:11:42.211103 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d9ac13ed-85c2-4a8e-974e-d1d28eb19012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:11:42.217385 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 743/1474] 10.222.0.16 () {58 vars in 1059 bytes} [Fri Jun 24 04:11:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:11:42.224559 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc37d31-3a9e-4f56-be68-2c35796f59e6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:11:42.235303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc37d31-3a9e-4f56-be68-2c35796f59e6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:11:42.238808 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a1514-9e64-4f2c-817b-fb009219a019 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61459e15d327405b88a2fdfc1fc0fecd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:11:42.239538 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a1514-9e64-4f2c-817b-fb009219a019 service glance] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:42.239829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a1514-9e64-4f2c-817b-fb009219a019 service glance] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:42.240109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a1514-9e64-4f2c-817b-fb009219a019 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:42.248983 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a1514-9e64-4f2c-817b-fb009219a019 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:11:42.250465 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a1514-9e64-4f2c-817b-fb009219a019 service glance] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:11:42.260315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 732/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:11:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:11:44.403378 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a2d9d-d8bc-4f42-9e05-f026806c4201 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:44.403936 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a2d9d-d8bc-4f42-9e05-f026806c4201 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:44.403936 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a2d9d-d8bc-4f42-9e05-f026806c4201 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:44.405341 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 744/1476] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:11:44.410585 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bacebc-a2d4-48e4-b213-bdd4f3d0d80f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:44.411016 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bacebc-a2d4-48e4-b213-bdd4f3d0d80f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:44.411147 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bacebc-a2d4-48e4-b213-bdd4f3d0d80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:44.429822 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c1bacebc-a2d4-48e4-b213-bdd4f3d0d80f None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:11:44.431117 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c1bacebc-a2d4-48e4-b213-bdd4f3d0d80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:11:44.438854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 733/1477] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:11:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:11:44.463153 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a2d9d-d8bc-4f42-9e05-f026806c4201 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:11:44.479584 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a2d9d-d8bc-4f42-9e05-f026806c4201 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:11:44.485479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c58809-7dc2-410b-8c9d-df6338afe5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:11:44.486122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c58809-7dc2-410b-8c9d-df6338afe5c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:44.486426 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c58809-7dc2-410b-8c9d-df6338afe5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:44.486705 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c58809-7dc2-410b-8c9d-df6338afe5c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:44.487439 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c58809-7dc2-410b-8c9d-df6338afe5c9 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:11:44.489195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-b7c58809-7dc2-410b-8c9d-df6338afe5c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 04:11:44.489195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: warnings.warn(msg) Jun 24 04:11:44.489195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: Jun 24 04:11:44.490390 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c58809-7dc2-410b-8c9d-df6338afe5c9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:11:44.511001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 745/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Jun 24 04:11:44 2022] GET /identity/v3/projects => generated 2553 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:11:46.586020 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1cea3-2d7e-444e-b2dd-adcd6cc80121 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:46.586613 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1cea3-2d7e-444e-b2dd-adcd6cc80121 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:46.586613 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1cea3-2d7e-444e-b2dd-adcd6cc80121 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:46.588004 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 734/1479] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:11:46.593140 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f6bbc-a242-46c9-b129-80e06997cb61 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:46.593551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f6bbc-a242-46c9-b129-80e06997cb61 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:46.593713 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f6bbc-a242-46c9-b129-80e06997cb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:46.611822 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-467f6bbc-a242-46c9-b129-80e06997cb61 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:11:46.613027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-467f6bbc-a242-46c9-b129-80e06997cb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:11:46.621447 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 746/1480] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:11:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:11:46.664763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1cea3-2d7e-444e-b2dd-adcd6cc80121 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:11:46.679636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1cea3-2d7e-444e-b2dd-adcd6cc80121 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:11:46.685038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f3cb1-a80e-42cf-afe2-d675a78d8a48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:11:46.685667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f3cb1-a80e-42cf-afe2-d675a78d8a48 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:46.685950 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f3cb1-a80e-42cf-afe2-d675a78d8a48 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:46.686227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f3cb1-a80e-42cf-afe2-d675a78d8a48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:46.694548 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f3cb1-a80e-42cf-afe2-d675a78d8a48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:11:46.696299 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f3cb1-a80e-42cf-afe2-d675a78d8a48 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:11:46.705594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 735/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:11:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:11:48.957753 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e176a27-8532-4614-8b48-3f82801b5cbd None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:48.958284 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e176a27-8532-4614-8b48-3f82801b5cbd None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:48.958404 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e176a27-8532-4614-8b48-3f82801b5cbd None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:48.959853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 747/1482] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:11:48.965401 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a1f62-a92b-4d78-9cc1-652123f3d5cb None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:48.965792 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a1f62-a92b-4d78-9cc1-652123f3d5cb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:48.965961 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a1f62-a92b-4d78-9cc1-652123f3d5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:48.983831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-888a1f62-a92b-4d78-9cc1-652123f3d5cb None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:11:48.985014 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-888a1f62-a92b-4d78-9cc1-652123f3d5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:11:48.993901 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 736/1483] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:11:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:11:49.029895 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e176a27-8532-4614-8b48-3f82801b5cbd None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:11:49.041129 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e176a27-8532-4614-8b48-3f82801b5cbd None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:11:49.044651 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9b32d-fda4-4a43-8c2f-b583062efcbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:11:49.045288 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9b32d-fda4-4a43-8c2f-b583062efcbb service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:49.045575 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9b32d-fda4-4a43-8c2f-b583062efcbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:49.045855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9b32d-fda4-4a43-8c2f-b583062efcbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:49.054978 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc9b32d-fda4-4a43-8c2f-b583062efcbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:11:49.056409 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc9b32d-fda4-4a43-8c2f-b583062efcbb service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:11:49.065879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 748/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:11:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:11:51.154572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0ab8a-c974-4fdb-b7fa-8fd06e0a47a5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:51.155124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0ab8a-c974-4fdb-b7fa-8fd06e0a47a5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:51.155124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0ab8a-c974-4fdb-b7fa-8fd06e0a47a5 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:51.156518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 737/1485] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:11:51.161950 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02132afc-f3b7-440c-b437-5747178c030d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:51.162407 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02132afc-f3b7-440c-b437-5747178c030d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:51.163111 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02132afc-f3b7-440c-b437-5747178c030d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:51.181450 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-02132afc-f3b7-440c-b437-5747178c030d None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:11:51.182706 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-02132afc-f3b7-440c-b437-5747178c030d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:11:51.191920 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 749/1486] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:11:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:11:51.231075 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0ab8a-c974-4fdb-b7fa-8fd06e0a47a5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:11:51.243964 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0ab8a-c974-4fdb-b7fa-8fd06e0a47a5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:11:51.247665 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349032b-d7f3-46de-a12c-8c49d2e78d2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:11:51.248305 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349032b-d7f3-46de-a12c-8c49d2e78d2f service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:51.248586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349032b-d7f3-46de-a12c-8c49d2e78d2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:51.248860 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349032b-d7f3-46de-a12c-8c49d2e78d2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:51.258419 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349032b-d7f3-46de-a12c-8c49d2e78d2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:11:51.259845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349032b-d7f3-46de-a12c-8c49d2e78d2f service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:11:51.269284 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 738/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:11:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:11:53.451604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3707f-2c55-4099-a4ff-af268bd7cc40 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:53.452140 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3707f-2c55-4099-a4ff-af268bd7cc40 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:53.452140 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3707f-2c55-4099-a4ff-af268bd7cc40 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:53.453517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 750/1488] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:11:53.459028 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54a433-5d2e-43b5-8d2b-9af70c78fe72 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:53.462136 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54a433-5d2e-43b5-8d2b-9af70c78fe72 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:53.462136 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54a433-5d2e-43b5-8d2b-9af70c78fe72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:53.477672 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9b54a433-5d2e-43b5-8d2b-9af70c78fe72 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:11:53.479322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9b54a433-5d2e-43b5-8d2b-9af70c78fe72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:11:53.486998 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 739/1489] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:11:53.530091 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f3707f-2c55-4099-a4ff-af268bd7cc40 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:11:53.540707 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f3707f-2c55-4099-a4ff-af268bd7cc40 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:11:53.544385 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4433cc5e-622c-4828-8852-9262348047d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:11:53.545058 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433cc5e-622c-4828-8852-9262348047d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:53.545347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433cc5e-622c-4828-8852-9262348047d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:53.545630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4433cc5e-622c-4828-8852-9262348047d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:53.554857 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433cc5e-622c-4828-8852-9262348047d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:11:53.562638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4433cc5e-622c-4828-8852-9262348047d2 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:11:53.565790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 751/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:11:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:11:55.948482 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b1941-a1f1-4ed9-94ee-b008a7395de5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:55.949056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b1941-a1f1-4ed9-94ee-b008a7395de5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:55.949056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b1941-a1f1-4ed9-94ee-b008a7395de5 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:55.950436 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 740/1491] 10.222.0.16 () {54 vars in 926 bytes} [Fri Jun 24 04:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:11:55.955532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45c7d0-3533-4d53-96f6-4848d2f3426b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:55.955976 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45c7d0-3533-4d53-96f6-4848d2f3426b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:55.956114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45c7d0-3533-4d53-96f6-4848d2f3426b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:55.974047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8f45c7d0-3533-4d53-96f6-4848d2f3426b None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:11:55.975882 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8f45c7d0-3533-4d53-96f6-4848d2f3426b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:11:55.983607 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 752/1492] 10.222.0.16 () {58 vars in 1023 bytes} [Fri Jun 24 04:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:11:56.022969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b1941-a1f1-4ed9-94ee-b008a7395de5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:11:56.033614 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b1941-a1f1-4ed9-94ee-b008a7395de5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:11:56.037113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20fc663-e95c-4447-a86c-038be132097f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:11:56.038541 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fc663-e95c-4447-a86c-038be132097f service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:11:56.038843 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fc663-e95c-4447-a86c-038be132097f service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:11:56.039125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fc663-e95c-4447-a86c-038be132097f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:11:56.048239 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20fc663-e95c-4447-a86c-038be132097f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:11:56.049678 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20fc663-e95c-4447-a86c-038be132097f service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:11:56.059260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 741/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:11:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:12:05.969164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=33 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:12:06.055085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=33 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:13:37.794944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ed0c8-4789-4adc-9085-d35d354b2435 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:13:37.795561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ed0c8-4789-4adc-9085-d35d354b2435 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:13:37.795561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ed0c8-4789-4adc-9085-d35d354b2435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:13:37.813657 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-903ed0c8-4789-4adc-9085-d35d354b2435 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:13:37.814880 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-903ed0c8-4789-4adc-9085-d35d354b2435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:13:37.823277 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 753/1494] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:13:37.832670 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a3ebf-46c1-4f66-8056-3db5b74ca1ce None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:13:37.833202 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a3ebf-46c1-4f66-8056-3db5b74ca1ce None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:13:37.833470 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a3ebf-46c1-4f66-8056-3db5b74ca1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:13:37.852219 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e77a3ebf-46c1-4f66-8056-3db5b74ca1ce None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:13:37.853457 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e77a3ebf-46c1-4f66-8056-3db5b74ca1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:13:37.861201 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 742/1495] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:13:37.866168 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ed0c8-4789-4adc-9085-d35d354b2435 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:13:37.875712 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ed0c8-4789-4adc-9085-d35d354b2435 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:13:37.881052 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16578ec-bb69-45c7-9e5e-26b63c246bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:13:37.881724 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16578ec-bb69-45c7-9e5e-26b63c246bba admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:13:37.881996 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16578ec-bb69-45c7-9e5e-26b63c246bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:13:37.882273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16578ec-bb69-45c7-9e5e-26b63c246bba admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:13:37.883039 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16578ec-bb69-45c7-9e5e-26b63c246bba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:13:37.883352 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16578ec-bb69-45c7-9e5e-26b63c246bba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:13:37.884372 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-b16578ec-bb69-45c7-9e5e-26b63c246bba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 04:13:37.884372 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: warnings.warn(msg) Jun 24 04:13:37.884372 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: Jun 24 04:13:37.885269 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16578ec-bb69-45c7-9e5e-26b63c246bba admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:13:37.897123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 754/1496] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:13:37.904910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a3ebf-46c1-4f66-8056-3db5b74ca1ce None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:13:37.914531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a3ebf-46c1-4f66-8056-3db5b74ca1ce None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:13:37.919051 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d75d2f-6fd7-4d59-b322-f3831dfe0b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:13:37.919724 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d75d2f-6fd7-4d59-b322-f3831dfe0b58 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:13:37.920008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d75d2f-6fd7-4d59-b322-f3831dfe0b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:13:37.920282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d75d2f-6fd7-4d59-b322-f3831dfe0b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:13:37.920996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d75d2f-6fd7-4d59-b322-f3831dfe0b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:13:37.922847 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-c5d75d2f-6fd7-4d59-b322-f3831dfe0b58 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 04:13:37.922847 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Jun 24 04:13:37.922847 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: Jun 24 04:13:37.924082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d75d2f-6fd7-4d59-b322-f3831dfe0b58 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:13:38.026434 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 743/1497] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:13:37 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:13:38.031078 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16578ec-bb69-45c7-9e5e-26b63c246bba admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:13:38.042253 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16578ec-bb69-45c7-9e5e-26b63c246bba admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:13:38.099354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5dd2f-d2cd-46af-abeb-2f98bc3e1b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:13:38.099935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5dd2f-d2cd-46af-abeb-2f98bc3e1b85 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:13:38.100222 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5dd2f-d2cd-46af-abeb-2f98bc3e1b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:13:38.100496 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5dd2f-d2cd-46af-abeb-2f98bc3e1b85 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:13:38.101188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd5dd2f-d2cd-46af-abeb-2f98bc3e1b85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:13:38.102203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-4dd5dd2f-d2cd-46af-abeb-2f98bc3e1b85 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 04:13:38.102203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: warnings.warn(msg) Jun 24 04:13:38.102203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: Jun 24 04:13:38.103466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd5dd2f-d2cd-46af-abeb-2f98bc3e1b85 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:13:38.231540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 755/1498] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:13:38 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:13:38.235650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d75d2f-6fd7-4d59-b322-f3831dfe0b58 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:13:38.246353 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d75d2f-6fd7-4d59-b322-f3831dfe0b58 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:13:38.250908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aeb72c-fde0-4ecc-96cc-d682a84a5ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:13:38.251560 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aeb72c-fde0-4ecc-96cc-d682a84a5ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:13:38.251846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aeb72c-fde0-4ecc-96cc-d682a84a5ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:13:38.252123 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aeb72c-fde0-4ecc-96cc-d682a84a5ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:13:38.252847 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aeb72c-fde0-4ecc-96cc-d682a84a5ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:13:38.253867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-07aeb72c-fde0-4ecc-96cc-d682a84a5ac1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 04:13:38.253867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Jun 24 04:13:38.253867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: Jun 24 04:13:38.254810 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aeb72c-fde0-4ecc-96cc-d682a84a5ac1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:13:38.267418 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 744/1499] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:13:38.272263 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5dd2f-d2cd-46af-abeb-2f98bc3e1b85 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:13:38.281931 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5dd2f-d2cd-46af-abeb-2f98bc3e1b85 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:13:38.287174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1137e-c8cb-4543-b212-24a6e6ac753f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:13:38.294039 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1137e-c8cb-4543-b212-24a6e6ac753f admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:13:38.294039 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1137e-c8cb-4543-b212-24a6e6ac753f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:13:38.294039 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1137e-c8cb-4543-b212-24a6e6ac753f admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:13:38.294039 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1137e-c8cb-4543-b212-24a6e6ac753f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:13:38.294039 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-ffc1137e-c8cb-4543-b212-24a6e6ac753f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 04:13:38.294039 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: warnings.warn(msg) Jun 24 04:13:38.294039 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: Jun 24 04:13:38.294709 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1137e-c8cb-4543-b212-24a6e6ac753f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:13:38.303446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 756/1500] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:13:38.307759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aeb72c-fde0-4ecc-96cc-d682a84a5ac1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:13:38.317402 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aeb72c-fde0-4ecc-96cc-d682a84a5ac1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:13:38.321858 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a0ee5-e7fe-46f8-bbf0-90ff622dd9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:13:38.322754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a0ee5-e7fe-46f8-bbf0-90ff622dd9c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:13:38.323036 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a0ee5-e7fe-46f8-bbf0-90ff622dd9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:13:38.323307 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a0ee5-e7fe-46f8-bbf0-90ff622dd9c3 admin admin] PATH_INFO: `/v3/projects/1cd01e55ccc84ca0886b71718cc2384c/users/3a3ebfed200041b79718b9a02f72a084/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:13:38.338209 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a0ee5-e7fe-46f8-bbf0-90ff622dd9c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd01e55ccc84ca0886b71718cc2384c, user_id=3a3ebfed200041b79718b9a02f72a084, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:13:38.339271 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-9d0a0ee5-e7fe-46f8-bbf0-90ff622dd9c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 04:13:38.339271 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Jun 24 04:13:38.339271 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: Jun 24 04:13:38.341186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a0ee5-e7fe-46f8-bbf0-90ff622dd9c3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:13:38.408695 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 745/1501] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:13:38 2022] PUT /identity/v3/projects/1cd01e55ccc84ca0886b71718cc2384c/users/3a3ebfed200041b79718b9a02f72a084/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:13:38.428043 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a38a0-7dbe-4d8e-b270-216470d0e145 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:13:38.428334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a38a0-7dbe-4d8e-b270-216470d0e145 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:13:38.428611 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a38a0-7dbe-4d8e-b270-216470d0e145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:13:38.455058 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-878a38a0-7dbe-4d8e-b270-216470d0e145 None None] MFA Rules not processed for user `3a3ebfed200041b79718b9a02f72a084`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:13:38.456268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-878a38a0-7dbe-4d8e-b270-216470d0e145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:13:38.518982 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 757/1502] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:13:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:13:38.532782 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b82843-9103-4458-93af-47b81a8a4a94 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:13:38.533076 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b82843-9103-4458-93af-47b81a8a4a94 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:13:38.533358 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b82843-9103-4458-93af-47b81a8a4a94 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:13:38.534750 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 746/1503] 10.222.0.16 () {54 vars in 963 bytes} [Fri Jun 24 04:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:13:38.539633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef133e0e-8626-45eb-aafe-95d9dbe60230 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:13:38.539926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef133e0e-8626-45eb-aafe-95d9dbe60230 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:13:38.540208 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef133e0e-8626-45eb-aafe-95d9dbe60230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:13:38.557883 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ef133e0e-8626-45eb-aafe-95d9dbe60230 None None] MFA Rules not processed for user `0fbec7435e0841a5b96946dca814e0c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:13:38.559366 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ef133e0e-8626-45eb-aafe-95d9dbe60230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:13:38.601584 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:13:38.608734 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 758/1504] 10.222.0.16 () {58 vars in 1060 bytes} [Fri Jun 24 04:13:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:13:38.617749 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b82843-9103-4458-93af-47b81a8a4a94 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:13:38.628216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b82843-9103-4458-93af-47b81a8a4a94 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:13:38.632757 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55bace2-c3e2-47fd-8390-371d99bff6c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:13:38.633382 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55bace2-c3e2-47fd-8390-371d99bff6c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:13:38.633681 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55bace2-c3e2-47fd-8390-371d99bff6c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:13:38.633959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55bace2-c3e2-47fd-8390-371d99bff6c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:13:38.642065 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55bace2-c3e2-47fd-8390-371d99bff6c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:13:38.643516 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55bace2-c3e2-47fd-8390-371d99bff6c1 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:13:38.652894 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 747/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:13:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:13:38.695207 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609537e9-2a28-47ad-8348-33082c6635e8 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:13:38.695500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609537e9-2a28-47ad-8348-33082c6635e8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:13:38.695783 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609537e9-2a28-47ad-8348-33082c6635e8 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:13:38.697198 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 759/1506] 10.222.0.16 () {52 vars in 829 bytes} [Fri Jun 24 04:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:13:38.700617 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55bace2-c3e2-47fd-8390-371d99bff6c1 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:13:38.762189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55bace2-c3e2-47fd-8390-371d99bff6c1 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:13:38.766682 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfe62f-2505-4933-a36d-4081103055a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:13:38.767546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfe62f-2505-4933-a36d-4081103055a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:13:38.767829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfe62f-2505-4933-a36d-4081103055a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:13:38.768107 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfe62f-2505-4933-a36d-4081103055a6 admin admin] PATH_INFO: `/v3/users/3a3ebfed200041b79718b9a02f72a084` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:13:38.769538 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfe62f-2505-4933-a36d-4081103055a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a3ebfed200041b79718b9a02f72a084)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:13:38.770779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-87dfe62f-2505-4933-a36d-4081103055a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 04:13:38.770779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Jun 24 04:13:38.770779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: Jun 24 04:13:38.771981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfe62f-2505-4933-a36d-4081103055a6 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:13:39.009820 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-87dfe62f-2505-4933-a36d-4081103055a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a3ebfed200041b79718b9a02f72a084'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:13:39.029816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-87dfe62f-2505-4933-a36d-4081103055a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a3ebfed200041b79718b9a02f72a084'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:13:39.105978 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-87dfe62f-2505-4933-a36d-4081103055a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a3ebfed200041b79718b9a02f72a084'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:13:39.106642 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-87dfe62f-2505-4933-a36d-4081103055a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a3ebfed200041b79718b9a02f72a084'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:13:39.107338 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-87dfe62f-2505-4933-a36d-4081103055a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a3ebfed200041b79718b9a02f72a084'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:13:39.116558 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:13:39.126171 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 748/1507] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:13:38 2022] DELETE /identity/v3/users/3a3ebfed200041b79718b9a02f72a084 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:13:39.130786 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba82a9f-ff15-4509-93d0-f485fdeb1873 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:13:39.131251 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba82a9f-ff15-4509-93d0-f485fdeb1873 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:13:39.131363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba82a9f-ff15-4509-93d0-f485fdeb1873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:13:39.152368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4ba82a9f-ff15-4509-93d0-f485fdeb1873 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:13:39.153600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4ba82a9f-ff15-4509-93d0-f485fdeb1873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:13:39.217951 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 760/1508] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:13:39.229498 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfe62f-2505-4933-a36d-4081103055a6 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:13:39.230960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-87dfe62f-2505-4933-a36d-4081103055a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:13:39.289358 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfe62f-2505-4933-a36d-4081103055a6 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:13:39.292858 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b68a44-7c0e-408c-89c1-c84d94b8ea41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:13:39.293505 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b68a44-7c0e-408c-89c1-c84d94b8ea41 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:13:39.293794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b68a44-7c0e-408c-89c1-c84d94b8ea41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:13:39.294073 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b68a44-7c0e-408c-89c1-c84d94b8ea41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:13:39.303068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b68a44-7c0e-408c-89c1-c84d94b8ea41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:13:39.304511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b68a44-7c0e-408c-89c1-c84d94b8ea41 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:13:39.313729 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 749/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:13:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:13:39.815400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba82a9f-ff15-4509-93d0-f485fdeb1873 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:13:39.816810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4ba82a9f-ff15-4509-93d0-f485fdeb1873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:13:39.833816 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba82a9f-ff15-4509-93d0-f485fdeb1873 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:13:39.838652 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6ced7-7ad5-4ba2-9ca6-dc19c39113ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:13:39.839529 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6ced7-7ad5-4ba2-9ca6-dc19c39113ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:13:39.839831 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6ced7-7ad5-4ba2-9ca6-dc19c39113ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:13:39.840102 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6ced7-7ad5-4ba2-9ca6-dc19c39113ae admin admin] PATH_INFO: `/v3/projects/1cd01e55ccc84ca0886b71718cc2384c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:13:39.841300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e6ced7-7ad5-4ba2-9ca6-dc19c39113ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cd01e55ccc84ca0886b71718cc2384c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:13:39.843295 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-93e6ced7-7ad5-4ba2-9ca6-dc19c39113ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 04:13:39.843295 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: warnings.warn(msg) Jun 24 04:13:39.843295 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: Jun 24 04:13:39.844854 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e6ced7-7ad5-4ba2-9ca6-dc19c39113ae admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:13:39.955949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-93e6ced7-7ad5-4ba2-9ca6-dc19c39113ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cd01e55ccc84ca0886b71718cc2384c', 'request_id': 'req-93e6ced7-7ad5-4ba2-9ca6-dc19c39113ae'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:13:39.969243 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-93e6ced7-7ad5-4ba2-9ca6-dc19c39113ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cd01e55ccc84ca0886b71718cc2384c', 'request_id': 'req-93e6ced7-7ad5-4ba2-9ca6-dc19c39113ae'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:13:39.975879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-93e6ced7-7ad5-4ba2-9ca6-dc19c39113ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cd01e55ccc84ca0886b71718cc2384c', 'request_id': 'req-93e6ced7-7ad5-4ba2-9ca6-dc19c39113ae'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:13:40.043138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-93e6ced7-7ad5-4ba2-9ca6-dc19c39113ae admin admin] The token cache is being invalidate because project 1cd01e55ccc84ca0886b71718cc2384c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:13:40.043138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-93e6ced7-7ad5-4ba2-9ca6-dc19c39113ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:13:40.043686 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-93e6ced7-7ad5-4ba2-9ca6-dc19c39113ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:13:40.045261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 761/1510] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:13:39 2022] DELETE /identity/v3/projects/1cd01e55ccc84ca0886b71718cc2384c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:13:49.309663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=29 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:13:50.039371 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:15:13.410795 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0be55e-28a1-40dd-a22c-c00bd18c718b None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:13.414112 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0be55e-28a1-40dd-a22c-c00bd18c718b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:13.414112 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0be55e-28a1-40dd-a22c-c00bd18c718b None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:13.414112 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 04:15:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:15:13.417531 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3e853-c3f2-4982-b3f9-9089f997945b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:13.417987 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3e853-c3f2-4982-b3f9-9089f997945b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:13.418135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3e853-c3f2-4982-b3f9-9089f997945b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:13.437375 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-baf3e853-c3f2-4982-b3f9-9089f997945b None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:15:13.438668 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-baf3e853-c3f2-4982-b3f9-9089f997945b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:15:13.498184 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 04:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:15:13.654584 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0be55e-28a1-40dd-a22c-c00bd18c718b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:15:13.660482 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3a0be55e-28a1-40dd-a22c-c00bd18c718b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:15:13.732393 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0be55e-28a1-40dd-a22c-c00bd18c718b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:15:13.737923 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145f309-3627-4f12-a514-81e7c04c8c7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:15:13.738595 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145f309-3627-4f12-a514-81e7c04c8c7e service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:13.738883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145f309-3627-4f12-a514-81e7c04c8c7e service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:13.739161 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145f309-3627-4f12-a514-81e7c04c8c7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:13.747144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145f309-3627-4f12-a514-81e7c04c8c7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:15:13.748721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145f309-3627-4f12-a514-81e7c04c8c7e service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:15:13.758792 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 751/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:15:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:15:14.600688 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aec7a0-f23f-41e4-a3f4-13c8beb2fc47 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:14.601268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aec7a0-f23f-41e4-a3f4-13c8beb2fc47 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:14.601268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aec7a0-f23f-41e4-a3f4-13c8beb2fc47 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:14.602677 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 04:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:15:14.607429 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e8663-3f98-4134-8f03-23b13a7acfdb None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:14.607867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e8663-3f98-4134-8f03-23b13a7acfdb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:14.607981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e8663-3f98-4134-8f03-23b13a7acfdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:14.627030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e50e8663-3f98-4134-8f03-23b13a7acfdb None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:15:14.629388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e50e8663-3f98-4134-8f03-23b13a7acfdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:15:14.637669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 04:15:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:15:14.792153 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aec7a0-f23f-41e4-a3f4-13c8beb2fc47 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:15:14.807036 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aec7a0-f23f-41e4-a3f4-13c8beb2fc47 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:15:14.812348 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd497e66-4973-4ffc-9aff-e79b52f69dcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:15:14.812981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd497e66-4973-4ffc-9aff-e79b52f69dcc service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:14.813265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd497e66-4973-4ffc-9aff-e79b52f69dcc service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:14.813540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd497e66-4973-4ffc-9aff-e79b52f69dcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:14.822200 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd497e66-4973-4ffc-9aff-e79b52f69dcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:15:14.823830 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd497e66-4973-4ffc-9aff-e79b52f69dcc service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:15:14.833063 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 764/1516] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:15:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:15:15.625564 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea48e06-a8f3-4682-b2f1-748a5621fa53 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:15.626115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea48e06-a8f3-4682-b2f1-748a5621fa53 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:15.626115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea48e06-a8f3-4682-b2f1-748a5621fa53 None None] PATH_INFO: `/v3` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:15.627493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 753/1517] 10.222.0.43 () {54 vars in 925 bytes} [Fri Jun 24 04:15:15 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:15:15.633844 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1a148-0fc6-4f17-b329-7374e951d12c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:15.634260 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1a148-0fc6-4f17-b329-7374e951d12c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:15.634433 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1a148-0fc6-4f17-b329-7374e951d12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:15.654863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-12e1a148-0fc6-4f17-b329-7374e951d12c None None] MFA Rules not processed for user `7662e61619d44c92af3ffacbacba3913`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:15:15.656075 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-12e1a148-0fc6-4f17-b329-7374e951d12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:15:15.717419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 765/1518] 10.222.0.43 () {58 vars in 1015 bytes} [Fri Jun 24 04:15:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:15:19.746728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf4e9b-deb2-4834-9637-772276d121b6 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:19.747196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf4e9b-deb2-4834-9637-772276d121b6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:19.747196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf4e9b-deb2-4834-9637-772276d121b6 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:19.748577 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 754/1519] 10.222.0.16 () {54 vars in 960 bytes} [Fri Jun 24 04:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:15:19.754788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02c85c-54b1-4a80-9e9f-8f100350b3f5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:19.755230 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02c85c-54b1-4a80-9e9f-8f100350b3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:19.755357 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02c85c-54b1-4a80-9e9f-8f100350b3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:19.774321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2b02c85c-54b1-4a80-9e9f-8f100350b3f5 None None] MFA Rules not processed for user `7662e61619d44c92af3ffacbacba3913`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:15:19.775559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2b02c85c-54b1-4a80-9e9f-8f100350b3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:15:19.784172 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 766/1520] 10.222.0.16 () {58 vars in 1057 bytes} [Fri Jun 24 04:15:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:15:19.792088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf4e9b-deb2-4834-9637-772276d121b6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:15:19.807718 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf4e9b-deb2-4834-9637-772276d121b6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:15:19.813143 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905755d-b6b6-42ab-9928-1388114d86f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7662e61619d44c92af3ffacbacba3913', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:15:19.813791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905755d-b6b6-42ab-9928-1388114d86f8 service placement] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:19.814070 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905755d-b6b6-42ab-9928-1388114d86f8 service placement] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:19.814346 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905755d-b6b6-42ab-9928-1388114d86f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:19.824170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905755d-b6b6-42ab-9928-1388114d86f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:15:19.825779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5905755d-b6b6-42ab-9928-1388114d86f8 service placement] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:15:19.836400 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 755/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:15:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:15:26.064116 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fdec9-7fb2-4996-9454-158b7f75388d None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:26.064611 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fdec9-7fb2-4996-9454-158b7f75388d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:26.064717 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fdec9-7fb2-4996-9454-158b7f75388d None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:26.066681 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 04:15:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:15:26.071462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51954ea4-ccee-4ca1-8a2e-0d8c612b3c38 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:26.071888 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51954ea4-ccee-4ca1-8a2e-0d8c612b3c38 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:26.072020 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51954ea4-ccee-4ca1-8a2e-0d8c612b3c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:26.091849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-51954ea4-ccee-4ca1-8a2e-0d8c612b3c38 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:15:26.093056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-51954ea4-ccee-4ca1-8a2e-0d8c612b3c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:15:26.101301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 04:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:15:26.264060 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fdec9-7fb2-4996-9454-158b7f75388d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:15:26.280506 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fdec9-7fb2-4996-9454-158b7f75388d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:15:26.288064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19131877-e106-4e17-9032-c5e8220212ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:15:26.288708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19131877-e106-4e17-9032-c5e8220212ad service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:26.288992 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19131877-e106-4e17-9032-c5e8220212ad service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:26.289268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19131877-e106-4e17-9032-c5e8220212ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:26.300335 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19131877-e106-4e17-9032-c5e8220212ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:15:26.300824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19131877-e106-4e17-9032-c5e8220212ad service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:15:26.312449 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 768/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:15:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:15:27.151334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fb86b-cc7f-48f8-b530-d23f8b19d780 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:27.151936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fb86b-cc7f-48f8-b530-d23f8b19d780 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:27.151936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fb86b-cc7f-48f8-b530-d23f8b19d780 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:27.153334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 04:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:15:27.158051 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5fee0-b7e0-443b-858d-a64a7fd65bd2 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:27.158470 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5fee0-b7e0-443b-858d-a64a7fd65bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:27.159167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5fee0-b7e0-443b-858d-a64a7fd65bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:27.178000 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-79d5fee0-b7e0-443b-858d-a64a7fd65bd2 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:15:27.179212 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-79d5fee0-b7e0-443b-858d-a64a7fd65bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:15:27.186864 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 04:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:15:27.340524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fb86b-cc7f-48f8-b530-d23f8b19d780 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:15:27.355732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fb86b-cc7f-48f8-b530-d23f8b19d780 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:15:27.361118 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815989b-37d0-4715-af8a-eaa927fc09a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:15:27.362465 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815989b-37d0-4715-af8a-eaa927fc09a3 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:27.362699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815989b-37d0-4715-af8a-eaa927fc09a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:27.362977 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815989b-37d0-4715-af8a-eaa927fc09a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:27.371255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815989b-37d0-4715-af8a-eaa927fc09a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:15:27.372896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3815989b-37d0-4715-af8a-eaa927fc09a3 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:15:27.382485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 758/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:15:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:15:31.255080 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccd1be-64c6-446c-add7-2a64f985b1bb None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:31.255680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccd1be-64c6-446c-add7-2a64f985b1bb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:31.255680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ccd1be-64c6-446c-add7-2a64f985b1bb None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:31.256994 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 04:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:15:31.262493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc68f86-f6ca-4137-940d-53fc85143ac9 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:31.262970 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc68f86-f6ca-4137-940d-53fc85143ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:31.263080 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc68f86-f6ca-4137-940d-53fc85143ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:31.281620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2bc68f86-f6ca-4137-940d-53fc85143ac9 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:15:31.282868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2bc68f86-f6ca-4137-940d-53fc85143ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:15:31.291306 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 04:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:15:31.327009 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccd1be-64c6-446c-add7-2a64f985b1bb None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:15:31.328701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d3ccd1be-64c6-446c-add7-2a64f985b1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:15:31.385915 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ccd1be-64c6-446c-add7-2a64f985b1bb None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:15:31.389473 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b28cb5-f43d-4f7a-93c8-2d6857cce305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:15:31.390122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b28cb5-f43d-4f7a-93c8-2d6857cce305 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:31.390448 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b28cb5-f43d-4f7a-93c8-2d6857cce305 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:31.390734 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b28cb5-f43d-4f7a-93c8-2d6857cce305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:31.400401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b28cb5-f43d-4f7a-93c8-2d6857cce305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:15:31.401845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b28cb5-f43d-4f7a-93c8-2d6857cce305 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:15:31.406799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:15:31.412208 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 771/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:15:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:15:33.582914 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334dd83-3e68-4ec6-9b2d-aaa62a9253ae None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:33.583699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334dd83-3e68-4ec6-9b2d-aaa62a9253ae None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:33.583699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334dd83-3e68-4ec6-9b2d-aaa62a9253ae None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:33.584826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 04:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:15:33.590020 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb869f4-f25a-4788-88c7-2f623ee0f36e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:33.590508 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb869f4-f25a-4788-88c7-2f623ee0f36e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:33.590626 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb869f4-f25a-4788-88c7-2f623ee0f36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:33.609208 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5eb869f4-f25a-4788-88c7-2f623ee0f36e None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:15:33.610337 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5eb869f4-f25a-4788-88c7-2f623ee0f36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:15:33.619092 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 04:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:15:33.655382 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334dd83-3e68-4ec6-9b2d-aaa62a9253ae None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:15:33.657060 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c334dd83-3e68-4ec6-9b2d-aaa62a9253ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:15:33.670673 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334dd83-3e68-4ec6-9b2d-aaa62a9253ae None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:15:33.675306 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b134c-1d75-4e95-ad1e-3a3b74eb1940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:15:33.675937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b134c-1d75-4e95-ad1e-3a3b74eb1940 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:33.676222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b134c-1d75-4e95-ad1e-3a3b74eb1940 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:33.676498 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b134c-1d75-4e95-ad1e-3a3b74eb1940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:33.685271 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b134c-1d75-4e95-ad1e-3a3b74eb1940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:15:33.686736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b134c-1d75-4e95-ad1e-3a3b74eb1940 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:15:33.696546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 761/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:15:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:15:37.169318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8929b-35ff-466f-9ab6-4cc1dc1fbf76 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:37.169908 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8929b-35ff-466f-9ab6-4cc1dc1fbf76 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:37.169908 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8929b-35ff-466f-9ab6-4cc1dc1fbf76 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:37.171263 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 04:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:15:37.176580 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc2f3e-7a03-48f8-b3bf-697beb2036d6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:37.176982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc2f3e-7a03-48f8-b3bf-697beb2036d6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:37.177136 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc2f3e-7a03-48f8-b3bf-697beb2036d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:37.195457 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b1cc2f3e-7a03-48f8-b3bf-697beb2036d6 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:15:37.196647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b1cc2f3e-7a03-48f8-b3bf-697beb2036d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:15:37.204338 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 04:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:15:37.505600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8929b-35ff-466f-9ab6-4cc1dc1fbf76 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:15:37.507299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0bd8929b-35ff-466f-9ab6-4cc1dc1fbf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:15:37.565392 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8929b-35ff-466f-9ab6-4cc1dc1fbf76 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:15:37.568926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80b552-b32c-458e-8ad9-7ed5c623f0be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:15:37.569575 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80b552-b32c-458e-8ad9-7ed5c623f0be service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:37.569858 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80b552-b32c-458e-8ad9-7ed5c623f0be service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:37.570128 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80b552-b32c-458e-8ad9-7ed5c623f0be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:37.579005 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80b552-b32c-458e-8ad9-7ed5c623f0be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:15:37.580641 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80b552-b32c-458e-8ad9-7ed5c623f0be service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:15:37.589900 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 774/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:15:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:15:39.675577 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80f46f-bf4f-4e7e-9543-6da0f3f2a440 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:39.676164 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80f46f-bf4f-4e7e-9543-6da0f3f2a440 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:39.676164 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80f46f-bf4f-4e7e-9543-6da0f3f2a440 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:39.677500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 04:15:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:15:39.682625 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129df4b3-0b5e-4622-85b1-1579e88ec8d7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:39.683065 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129df4b3-0b5e-4622-85b1-1579e88ec8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:39.683169 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129df4b3-0b5e-4622-85b1-1579e88ec8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:39.701556 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-129df4b3-0b5e-4622-85b1-1579e88ec8d7 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:15:39.702775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-129df4b3-0b5e-4622-85b1-1579e88ec8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:15:39.711635 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 04:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:15:40.020523 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80f46f-bf4f-4e7e-9543-6da0f3f2a440 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:15:40.022158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3e80f46f-bf4f-4e7e-9543-6da0f3f2a440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:15:40.030141 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=65 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:15:40.039438 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e80f46f-bf4f-4e7e-9543-6da0f3f2a440 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:15:40.043457 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6403c92-08e8-4390-afc4-48636d90a369 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:15:40.044162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6403c92-08e8-4390-afc4-48636d90a369 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:15:40.044162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6403c92-08e8-4390-afc4-48636d90a369 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:15:40.044342 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6403c92-08e8-4390-afc4-48636d90a369 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:15:40.053686 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6403c92-08e8-4390-afc4-48636d90a369 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:15:40.055149 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6403c92-08e8-4390-afc4-48636d90a369 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:15:40.065409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 764/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:15:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:15:49.696679 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=32 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:15:50.065661 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:09.430539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa87c8-d287-41cb-a6cc-732bdc28c322 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:09.436223 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa87c8-d287-41cb-a6cc-732bdc28c322 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:09.436223 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa87c8-d287-41cb-a6cc-732bdc28c322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:09.462665 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-58fa87c8-d287-41cb-a6cc-732bdc28c322 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:09.463875 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-58fa87c8-d287-41cb-a6cc-732bdc28c322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:09.477386 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 776/1540] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:17:09.487653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ae8b3-e99b-481a-b36a-142372a731e1 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:09.488242 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ae8b3-e99b-481a-b36a-142372a731e1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:09.488242 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ae8b3-e99b-481a-b36a-142372a731e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:09.519697 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ee4ae8b3-e99b-481a-b36a-142372a731e1 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:09.521226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ee4ae8b3-e99b-481a-b36a-142372a731e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:09.530003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 765/1541] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:09.535390 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa87c8-d287-41cb-a6cc-732bdc28c322 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:09.545815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa87c8-d287-41cb-a6cc-732bdc28c322 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:09.555074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971c62f-67a2-4bda-bf0f-4914eb96ab3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:09.555779 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971c62f-67a2-4bda-bf0f-4914eb96ab3d admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:09.556066 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971c62f-67a2-4bda-bf0f-4914eb96ab3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:09.556348 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971c62f-67a2-4bda-bf0f-4914eb96ab3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:09.557105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971c62f-67a2-4bda-bf0f-4914eb96ab3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:17:09.557421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971c62f-67a2-4bda-bf0f-4914eb96ab3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:09.559163 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8971c62f-67a2-4bda-bf0f-4914eb96ab3d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:09.579499 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 777/1542] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:17:09.587807 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ae8b3-e99b-481a-b36a-142372a731e1 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:09.597961 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ae8b3-e99b-481a-b36a-142372a731e1 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:09.606313 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71835ea-feb5-4b79-ae03-fac58fdc7f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:09.607027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71835ea-feb5-4b79-ae03-fac58fdc7f2f admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:09.607314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71835ea-feb5-4b79-ae03-fac58fdc7f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:09.607594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71835ea-feb5-4b79-ae03-fac58fdc7f2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:09.608303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71835ea-feb5-4b79-ae03-fac58fdc7f2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:09.613281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71835ea-feb5-4b79-ae03-fac58fdc7f2f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:09.747846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 766/1543] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:17:09 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:09.751507 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971c62f-67a2-4bda-bf0f-4914eb96ab3d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:09.762083 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971c62f-67a2-4bda-bf0f-4914eb96ab3d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:09.821561 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c185aac-0597-4fb1-8298-46e6b551485a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:09.823330 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c185aac-0597-4fb1-8298-46e6b551485a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:09.823330 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c185aac-0597-4fb1-8298-46e6b551485a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:09.858362 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-5c185aac-0597-4fb1-8298-46e6b551485a None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:09.865357 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5c185aac-0597-4fb1-8298-46e6b551485a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:09.868476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69eff2-fe97-4b6b-a47f-55c52f26156e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:09.869209 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69eff2-fe97-4b6b-a47f-55c52f26156e admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:09.869495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69eff2-fe97-4b6b-a47f-55c52f26156e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:09.869774 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69eff2-fe97-4b6b-a47f-55c52f26156e admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:09.870542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69eff2-fe97-4b6b-a47f-55c52f26156e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:09.872728 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69eff2-fe97-4b6b-a47f-55c52f26156e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:09.893206 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 767/1544] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:17:09.894717 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c83cb-4281-4abe-8174-871b3a11c90f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:09.895012 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c83cb-4281-4abe-8174-871b3a11c90f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:09.895294 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c83cb-4281-4abe-8174-871b3a11c90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:09.923629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-635c83cb-4281-4abe-8174-871b3a11c90f None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:09.924871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-635c83cb-4281-4abe-8174-871b3a11c90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:09.938353 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 768/1545] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:09.939766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a097577-48f0-4c5a-9bd1-866552783366 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:09.940057 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a097577-48f0-4c5a-9bd1-866552783366 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:09.940335 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a097577-48f0-4c5a-9bd1-866552783366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:09.967389 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1a097577-48f0-4c5a-9bd1-866552783366 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:09.968837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1a097577-48f0-4c5a-9bd1-866552783366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:09.977659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 778/1546] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:17:09 2022] POST /identity/v3/users => generated 288 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:09.979006 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695622c-55f3-4fd5-b1b6-8960927c5321 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:09.979285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695622c-55f3-4fd5-b1b6-8960927c5321 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:09.979566 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695622c-55f3-4fd5-b1b6-8960927c5321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:09.998253 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 769/1547] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:10.001330 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a097577-48f0-4c5a-9bd1-866552783366 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:10.015017 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8695622c-55f3-4fd5-b1b6-8960927c5321 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:10.016754 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8695622c-55f3-4fd5-b1b6-8960927c5321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:10.021307 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a097577-48f0-4c5a-9bd1-866552783366 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:10.027408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef508166-4d63-4e90-9dd0-e7e1bf847174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:10.028102 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef508166-4d63-4e90-9dd0-e7e1bf847174 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.028905 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef508166-4d63-4e90-9dd0-e7e1bf847174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.029426 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef508166-4d63-4e90-9dd0-e7e1bf847174 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.030181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef508166-4d63-4e90-9dd0-e7e1bf847174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:10.031918 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef508166-4d63-4e90-9dd0-e7e1bf847174 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:10.045052 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 779/1548] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:10.045933 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695622c-55f3-4fd5-b1b6-8960927c5321 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:10.059642 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 770/1549] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:10.060807 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6916d-de1d-42bc-8598-0b3937a23d0a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.061098 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6916d-de1d-42bc-8598-0b3937a23d0a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.061277 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695622c-55f3-4fd5-b1b6-8960927c5321 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:10.061446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6916d-de1d-42bc-8598-0b3937a23d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.067868 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94494f-d061-463a-ac78-532697b9ac6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:10.068931 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94494f-d061-463a-ac78-532697b9ac6e admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.069299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94494f-d061-463a-ac78-532697b9ac6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.069658 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94494f-d061-463a-ac78-532697b9ac6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.070733 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94494f-d061-463a-ac78-532697b9ac6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:17:10.071482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94494f-d061-463a-ac78-532697b9ac6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:10.074358 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94494f-d061-463a-ac78-532697b9ac6e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:10.083577 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-60f6916d-de1d-42bc-8598-0b3937a23d0a None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:10.084793 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-60f6916d-de1d-42bc-8598-0b3937a23d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:10.090467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 780/1550] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:17:10.092169 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb0ca2-8bce-4a21-abbc-4555e3a2a536 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.092470 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb0ca2-8bce-4a21-abbc-4555e3a2a536 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.092759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb0ca2-8bce-4a21-abbc-4555e3a2a536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.095090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 771/1551] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:10.096584 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca517bf-032b-407e-86d0-c3fdc4c99063 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.096873 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca517bf-032b-407e-86d0-c3fdc4c99063 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.097157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca517bf-032b-407e-86d0-c3fdc4c99063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.117417 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-cca517bf-032b-407e-86d0-c3fdc4c99063 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:10.117770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-75cb0ca2-8bce-4a21-abbc-4555e3a2a536 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:10.118936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-cca517bf-032b-407e-86d0-c3fdc4c99063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:10.119121 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-75cb0ca2-8bce-4a21-abbc-4555e3a2a536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:10.128029 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 772/1552] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:10.128893 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca517bf-032b-407e-86d0-c3fdc4c99063 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:10.129671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 781/1553] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:10.130685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb0ca2-8bce-4a21-abbc-4555e3a2a536 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:10.141897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca517bf-032b-407e-86d0-c3fdc4c99063 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:10.143558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb0ca2-8bce-4a21-abbc-4555e3a2a536 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:10.143702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a93ea-c413-4a62-805d-50c869dd4936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:10.144354 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a93ea-c413-4a62-805d-50c869dd4936 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.144640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a93ea-c413-4a62-805d-50c869dd4936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.144923 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a93ea-c413-4a62-805d-50c869dd4936 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.145653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a93ea-c413-4a62-805d-50c869dd4936 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:17:10.146225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a93ea-c413-4a62-805d-50c869dd4936 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:10.147292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-453a93ea-c413-4a62-805d-50c869dd4936 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 04:17:10.147292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Jun 24 04:17:10.147292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: Jun 24 04:17:10.148182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a93ea-c413-4a62-805d-50c869dd4936 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:10.148424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589d287-df1a-4c12-9a4f-b81100a809f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:10.149097 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589d287-df1a-4c12-9a4f-b81100a809f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.149438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589d287-df1a-4c12-9a4f-b81100a809f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.149850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589d287-df1a-4c12-9a4f-b81100a809f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.150609 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589d287-df1a-4c12-9a4f-b81100a809f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:10.152274 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589d287-df1a-4c12-9a4f-b81100a809f9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:10.160897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 773/1554] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:17:10.165879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a93ea-c413-4a62-805d-50c869dd4936 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:10.166109 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 782/1555] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:10.166109 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696784d3-ac5e-4c03-91bf-d4880446a484 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.166637 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696784d3-ac5e-4c03-91bf-d4880446a484 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.166819 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696784d3-ac5e-4c03-91bf-d4880446a484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.173004 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a93ea-c413-4a62-805d-50c869dd4936 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:10.177559 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483eebfa-fe88-4191-9447-c068f755d1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:10.178210 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eebfa-fe88-4191-9447-c068f755d1f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.178530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eebfa-fe88-4191-9447-c068f755d1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.178814 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eebfa-fe88-4191-9447-c068f755d1f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.179518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483eebfa-fe88-4191-9447-c068f755d1f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:10.181482 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483eebfa-fe88-4191-9447-c068f755d1f7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:10.283336 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 774/1556] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:17:10 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:10.284896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ac003-7046-4d52-991b-5bc4e7a7369e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.285196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ac003-7046-4d52-991b-5bc4e7a7369e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.285471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ac003-7046-4d52-991b-5bc4e7a7369e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.302539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-696784d3-ac5e-4c03-91bf-d4880446a484 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:10.303755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-696784d3-ac5e-4c03-91bf-d4880446a484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:10.304186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6e2ac003-7046-4d52-991b-5bc4e7a7369e None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:10.305340 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6e2ac003-7046-4d52-991b-5bc4e7a7369e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:10.367800 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 775/1557] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:10.369215 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da966960-b702-451c-87f5-03f9088b9422 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.369506 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da966960-b702-451c-87f5-03f9088b9422 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.369786 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da966960-b702-451c-87f5-03f9088b9422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.384667 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 783/1558] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:10.385852 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf990d-ce5b-4a6e-a6d5-f1071f799f74 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.386129 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf990d-ce5b-4a6e-a6d5-f1071f799f74 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.386432 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf990d-ce5b-4a6e-a6d5-f1071f799f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.389661 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-da966960-b702-451c-87f5-03f9088b9422 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:10.392013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-da966960-b702-451c-87f5-03f9088b9422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:10.400398 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 776/1559] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:10.401685 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69cc49-bc01-4d18-86a2-44ee1e4b462a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.401968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69cc49-bc01-4d18-86a2-44ee1e4b462a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.402246 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69cc49-bc01-4d18-86a2-44ee1e4b462a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.405037 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f1cf990d-ce5b-4a6e-a6d5-f1071f799f74 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:10.412678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f1cf990d-ce5b-4a6e-a6d5-f1071f799f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:10.421008 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 784/1560] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:10.421886 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf990d-ce5b-4a6e-a6d5-f1071f799f74 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:10.422030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ef69cc49-bc01-4d18-86a2-44ee1e4b462a None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:10.423211 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ef69cc49-bc01-4d18-86a2-44ee1e4b462a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:10.431364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf990d-ce5b-4a6e-a6d5-f1071f799f74 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:10.431878 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 777/1561] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:10.432595 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69cc49-bc01-4d18-86a2-44ee1e4b462a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:10.440064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f7109-2f77-426f-b69f-1311d6eaf088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:10.440957 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f7109-2f77-426f-b69f-1311d6eaf088 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.441237 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f7109-2f77-426f-b69f-1311d6eaf088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.441510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f7109-2f77-426f-b69f-1311d6eaf088 admin admin] PATH_INFO: `/v3/projects/cf6ac8a37c564019bdad3a570e9241f7/users/b938baf805604bd9ab55d1c364daf426/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.442722 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69cc49-bc01-4d18-86a2-44ee1e4b462a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:10.447291 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478c6470-ce2c-4622-9347-f8ddc8e75f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:10.447936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c6470-ce2c-4622-9347-f8ddc8e75f67 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.448222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c6470-ce2c-4622-9347-f8ddc8e75f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.448502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478c6470-ce2c-4622-9347-f8ddc8e75f67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.449199 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c6470-ce2c-4622-9347-f8ddc8e75f67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:10.451178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478c6470-ce2c-4622-9347-f8ddc8e75f67 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:10.456977 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f7109-2f77-426f-b69f-1311d6eaf088 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf6ac8a37c564019bdad3a570e9241f7, user_id=b938baf805604bd9ab55d1c364daf426, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:10.458010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-978f7109-2f77-426f-b69f-1311d6eaf088 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 04:17:10.458010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: warnings.warn(msg) Jun 24 04:17:10.458010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: Jun 24 04:17:10.460377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978f7109-2f77-426f-b69f-1311d6eaf088 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:10.531603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 785/1562] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:10 2022] PUT /identity/v3/projects/cf6ac8a37c564019bdad3a570e9241f7/users/b938baf805604bd9ab55d1c364daf426/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:10.532502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f7109-2f77-426f-b69f-1311d6eaf088 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:10.536670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:10.567931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 778/1563] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:17:10 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:10.575570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bc0a9-97aa-430a-a7f8-37196a0ac5cf None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.575570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bc0a9-97aa-430a-a7f8-37196a0ac5cf None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.575570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318bc0a9-97aa-430a-a7f8-37196a0ac5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.588334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-318bc0a9-97aa-430a-a7f8-37196a0ac5cf None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:10.589600 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-318bc0a9-97aa-430a-a7f8-37196a0ac5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:10.597828 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f7109-2f77-426f-b69f-1311d6eaf088 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:10.602165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:10.650674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 779/1564] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:10.651792 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318bc0a9-97aa-430a-a7f8-37196a0ac5cf None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:10.660236 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b14e6-26cb-4469-8b7b-c04bf9040502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:10.660916 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b14e6-26cb-4469-8b7b-c04bf9040502 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.661204 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b14e6-26cb-4469-8b7b-c04bf9040502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.661475 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318bc0a9-97aa-430a-a7f8-37196a0ac5cf None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:10.661596 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b14e6-26cb-4469-8b7b-c04bf9040502 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.662172 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b14e6-26cb-4469-8b7b-c04bf9040502 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:10.664284 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b14e6-26cb-4469-8b7b-c04bf9040502 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:10.666065 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a3f286-2372-421e-866d-6feca4ecf580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:10.666748 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3f286-2372-421e-866d-6feca4ecf580 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.667038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3f286-2372-421e-866d-6feca4ecf580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.667317 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3f286-2372-421e-866d-6feca4ecf580 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.668048 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3f286-2372-421e-866d-6feca4ecf580 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:17:10.668360 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3f286-2372-421e-866d-6feca4ecf580 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:10.669955 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a3f286-2372-421e-866d-6feca4ecf580 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:10.682205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 780/1565] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:17:10.683545 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7051c-e6cb-4999-9ea7-753f268d69e8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.687795 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7051c-e6cb-4999-9ea7-753f268d69e8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.687795 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7051c-e6cb-4999-9ea7-753f268d69e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.705139 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-bdf7051c-e6cb-4999-9ea7-753f268d69e8 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:10.706361 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bdf7051c-e6cb-4999-9ea7-753f268d69e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:10.715543 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 781/1566] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:10.716351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7051c-e6cb-4999-9ea7-753f268d69e8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:10.728811 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7051c-e6cb-4999-9ea7-753f268d69e8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:10.733471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd8638-7f39-4dee-b2a4-fd7edef592eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:10.734414 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd8638-7f39-4dee-b2a4-fd7edef592eb admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.734706 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd8638-7f39-4dee-b2a4-fd7edef592eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.735010 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd8638-7f39-4dee-b2a4-fd7edef592eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.735738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd8638-7f39-4dee-b2a4-fd7edef592eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:17:10.736099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd8638-7f39-4dee-b2a4-fd7edef592eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:10.737691 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd8638-7f39-4dee-b2a4-fd7edef592eb admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:10.743710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 786/1567] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:17:10 2022] POST /identity/v3/users => generated 294 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:10.744704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b14e6-26cb-4469-8b7b-c04bf9040502 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:10.753075 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 782/1568] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:17:10.754690 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b14e6-26cb-4469-8b7b-c04bf9040502 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:10.754922 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d3f34-8474-4989-be1f-04c58fa7a295 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.755217 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d3f34-8474-4989-be1f-04c58fa7a295 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.755500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d3f34-8474-4989-be1f-04c58fa7a295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.759297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011da34-f294-4b7f-85a1-a59b95e9f99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:10.759949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011da34-f294-4b7f-85a1-a59b95e9f99c admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.760232 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011da34-f294-4b7f-85a1-a59b95e9f99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.760510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3011da34-f294-4b7f-85a1-a59b95e9f99c admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.761233 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011da34-f294-4b7f-85a1-a59b95e9f99c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:17:10.761539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011da34-f294-4b7f-85a1-a59b95e9f99c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:10.763189 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3011da34-f294-4b7f-85a1-a59b95e9f99c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:10.777127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 787/1569] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:17:10.777923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011da34-f294-4b7f-85a1-a59b95e9f99c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:10.785562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c53d3f34-8474-4989-be1f-04c58fa7a295 None None] MFA Rules not processed for user `b938baf805604bd9ab55d1c364daf426`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:10.788837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c53d3f34-8474-4989-be1f-04c58fa7a295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:10.789778 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3011da34-f294-4b7f-85a1-a59b95e9f99c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:10.795417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb0acc-dc42-4398-b1ee-ea1eec80db66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:10.796098 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb0acc-dc42-4398-b1ee-ea1eec80db66 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.796826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb0acc-dc42-4398-b1ee-ea1eec80db66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.797129 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb0acc-dc42-4398-b1ee-ea1eec80db66 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.797837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb0acc-dc42-4398-b1ee-ea1eec80db66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:10.799845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb0acc-dc42-4398-b1ee-ea1eec80db66 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:10.850674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 783/1570] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:10.851512 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d3f34-8474-4989-be1f-04c58fa7a295 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:10.861586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d3f34-8474-4989-be1f-04c58fa7a295 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:10.866005 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378ed20-2669-4476-9b52-1bc67994facd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:10.866678 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378ed20-2669-4476-9b52-1bc67994facd admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.866963 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378ed20-2669-4476-9b52-1bc67994facd admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.867238 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378ed20-2669-4476-9b52-1bc67994facd admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.868009 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378ed20-2669-4476-9b52-1bc67994facd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:17:10.868315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378ed20-2669-4476-9b52-1bc67994facd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:10.869987 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378ed20-2669-4476-9b52-1bc67994facd admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:10.884494 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 788/1571] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:17:10 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:10.884494 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb0acc-dc42-4398-b1ee-ea1eec80db66 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:10.888619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 784/1572] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:17:10.889546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378ed20-2669-4476-9b52-1bc67994facd admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:10.894961 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb0acc-dc42-4398-b1ee-ea1eec80db66 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:10.899896 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce2d6b-5cd7-48d6-ac46-8b02296bdc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:10.900548 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce2d6b-5cd7-48d6-ac46-8b02296bdc71 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.900828 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce2d6b-5cd7-48d6-ac46-8b02296bdc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.901095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce2d6b-5cd7-48d6-ac46-8b02296bdc71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.901786 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce2d6b-5cd7-48d6-ac46-8b02296bdc71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:10.901972 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378ed20-2669-4476-9b52-1bc67994facd admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:10.902881 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-66ce2d6b-5cd7-48d6-ac46-8b02296bdc71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 04:17:10.902881 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: warnings.warn(msg) Jun 24 04:17:10.902881 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: Jun 24 04:17:10.904131 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce2d6b-5cd7-48d6-ac46-8b02296bdc71 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:10.907024 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716307b-1711-4100-9447-4ed678a31079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:10.907852 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716307b-1711-4100-9447-4ed678a31079 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.908176 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716307b-1711-4100-9447-4ed678a31079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.908492 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716307b-1711-4100-9447-4ed678a31079 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.909215 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c716307b-1711-4100-9447-4ed678a31079 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:17:10.909539 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c716307b-1711-4100-9447-4ed678a31079 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:10.911181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c716307b-1711-4100-9447-4ed678a31079 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:10.923957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 785/1573] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:17:10.924821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716307b-1711-4100-9447-4ed678a31079 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:10.934264 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716307b-1711-4100-9447-4ed678a31079 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:10.939438 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc78af-9987-478a-a4f8-e73b322ed196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:10.940083 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc78af-9987-478a-a4f8-e73b322ed196 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.940370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc78af-9987-478a-a4f8-e73b322ed196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.940651 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc78af-9987-478a-a4f8-e73b322ed196 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.941370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc78af-9987-478a-a4f8-e73b322ed196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:10.943080 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc78af-9987-478a-a4f8-e73b322ed196 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:10.952871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 786/1574] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:10.953627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc78af-9987-478a-a4f8-e73b322ed196 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:10.962800 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc78af-9987-478a-a4f8-e73b322ed196 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:10.967341 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e01ff-93c4-478b-9526-10d158e35522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:10.972011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e01ff-93c4-478b-9526-10d158e35522 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:10.972011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e01ff-93c4-478b-9526-10d158e35522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:10.972011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e01ff-93c4-478b-9526-10d158e35522 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:10.972011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e01ff-93c4-478b-9526-10d158e35522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:10.972011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e01ff-93c4-478b-9526-10d158e35522 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:10.993335 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 789/1575] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:17:10 2022] POST /identity/v3/projects => generated 363 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:10.994246 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce2d6b-5cd7-48d6-ac46-8b02296bdc71 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:11.047987 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 787/1576] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:17:10 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:11.049560 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c6f30-5897-469c-bc58-9f36a23ebc13 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:11.049853 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c6f30-5897-469c-bc58-9f36a23ebc13 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:11.050129 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c6f30-5897-469c-bc58-9f36a23ebc13 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:11.050487 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:11.051941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 788/1577] 10.222.0.16 () {54 vars in 961 bytes} [Fri Jun 24 04:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:17:11.052774 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c6f30-5897-469c-bc58-9f36a23ebc13 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:11.065165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce2d6b-5cd7-48d6-ac46-8b02296bdc71 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:11.071428 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:11.118193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c6f30-5897-469c-bc58-9f36a23ebc13 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:11.122159 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae86d4-1a9b-4725-88d3-d48e2070e24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:11.122861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae86d4-1a9b-4725-88d3-d48e2070e24d admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:11.122994 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b61b0c-2d45-4b2c-8b78-daab4f786f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:11.123151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae86d4-1a9b-4725-88d3-d48e2070e24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:11.123431 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae86d4-1a9b-4725-88d3-d48e2070e24d admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:11.123579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b61b0c-2d45-4b2c-8b78-daab4f786f77 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:11.123861 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b61b0c-2d45-4b2c-8b78-daab4f786f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:11.124128 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae86d4-1a9b-4725-88d3-d48e2070e24d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:11.124252 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b61b0c-2d45-4b2c-8b78-daab4f786f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:11.125099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b61b0c-2d45-4b2c-8b78-daab4f786f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:11.126116 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae86d4-1a9b-4725-88d3-d48e2070e24d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:11.126721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b61b0c-2d45-4b2c-8b78-daab4f786f77 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:11.138437 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 789/1578] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:11.139803 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b61b0c-2d45-4b2c-8b78-daab4f786f77 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:11.150958 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b61b0c-2d45-4b2c-8b78-daab4f786f77 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:11.156191 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b1706-eb4b-49d8-9313-d380d59cbbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:11.156912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b1706-eb4b-49d8-9313-d380d59cbbe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:11.157197 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b1706-eb4b-49d8-9313-d380d59cbbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:11.157479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b1706-eb4b-49d8-9313-d380d59cbbe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:11.158168 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b1706-eb4b-49d8-9313-d380d59cbbe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:11.160111 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b1706-eb4b-49d8-9313-d380d59cbbe5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:11.190100 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 790/1579] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:17:10 2022] POST /identity/v3/projects => generated 375 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:11.190955 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae86d4-1a9b-4725-88d3-d48e2070e24d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:11.252120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae86d4-1a9b-4725-88d3-d48e2070e24d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:11.256764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d482c2-1f04-4667-a1b6-36fca2192a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:11.257424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d482c2-1f04-4667-a1b6-36fca2192a39 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:11.257709 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d482c2-1f04-4667-a1b6-36fca2192a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:11.257989 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d482c2-1f04-4667-a1b6-36fca2192a39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:11.258710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d482c2-1f04-4667-a1b6-36fca2192a39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:11.260624 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d482c2-1f04-4667-a1b6-36fca2192a39 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:11.266085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 790/1580] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:17:11 2022] POST /identity/v3/projects => generated 393 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:11.269499 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b1706-eb4b-49d8-9313-d380d59cbbe5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:11.330640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b1706-eb4b-49d8-9313-d380d59cbbe5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:11.335387 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62d0eb-5e81-4c0b-a65f-bc4c1bc3e648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:11.336984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62d0eb-5e81-4c0b-a65f-bc4c1bc3e648 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:11.336984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62d0eb-5e81-4c0b-a65f-bc4c1bc3e648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:11.336984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62d0eb-5e81-4c0b-a65f-bc4c1bc3e648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:11.337351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b62d0eb-5e81-4c0b-a65f-bc4c1bc3e648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:11.338688 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 791/1581] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:17:11 2022] POST /identity/v3/projects => generated 367 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:11.339027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b62d0eb-5e81-4c0b-a65f-bc4c1bc3e648 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:11.339619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d482c2-1f04-4667-a1b6-36fca2192a39 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:11.349664 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 791/1582] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:11.350491 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62d0eb-5e81-4c0b-a65f-bc4c1bc3e648 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:11.361102 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62d0eb-5e81-4c0b-a65f-bc4c1bc3e648 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:11.405020 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d482c2-1f04-4667-a1b6-36fca2192a39 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:11.410625 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18159a-8d62-4fcb-8128-680c8d3c01cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:11.411290 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18159a-8d62-4fcb-8128-680c8d3c01cf admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:11.411580 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18159a-8d62-4fcb-8128-680c8d3c01cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:11.411854 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18159a-8d62-4fcb-8128-680c8d3c01cf admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:11.412538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18159a-8d62-4fcb-8128-680c8d3c01cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:11.414480 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18159a-8d62-4fcb-8128-680c8d3c01cf admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:11.417971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d83353-8c40-4f44-9e14-2147c0262e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:11.418675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d83353-8c40-4f44-9e14-2147c0262e40 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:11.418967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d83353-8c40-4f44-9e14-2147c0262e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:11.419253 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d83353-8c40-4f44-9e14-2147c0262e40 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:11.419947 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d83353-8c40-4f44-9e14-2147c0262e40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:11.420964 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-a5d83353-8c40-4f44-9e14-2147c0262e40 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 04:17:11.420964 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Jun 24 04:17:11.420964 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: Jun 24 04:17:11.422859 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d83353-8c40-4f44-9e14-2147c0262e40 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:11.423439 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:11.532375 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 792/1583] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:17:11 2022] POST /identity/v3/users => generated 290 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:11.532723 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 792/1584] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:17:11 2022] POST /identity/v3/users => generated 282 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:11.533667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e922054-0bc9-4ff4-90e4-e5e458496973 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:11.533781 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18159a-8d62-4fcb-8128-680c8d3c01cf admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:11.534042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e922054-0bc9-4ff4-90e4-e5e458496973 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:11.534224 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e922054-0bc9-4ff4-90e4-e5e458496973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:11.543965 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18159a-8d62-4fcb-8128-680c8d3c01cf admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:11.550275 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef8c80-1fd8-4d64-a140-bbcc5f47c7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:11.551011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef8c80-1fd8-4d64-a140-bbcc5f47c7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:11.551328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef8c80-1fd8-4d64-a140-bbcc5f47c7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:11.551613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaef8c80-1fd8-4d64-a140-bbcc5f47c7c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:11.552409 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef8c80-1fd8-4d64-a140-bbcc5f47c7c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:11.554081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaef8c80-1fd8-4d64-a140-bbcc5f47c7c1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:11.557264 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-5e922054-0bc9-4ff4-90e4-e5e458496973 None None] MFA Rules not processed for user `80b7b3ec60db4de19bbf7d08f111f470`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:11.558512 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5e922054-0bc9-4ff4-90e4-e5e458496973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:11.569493 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 793/1585] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:11.569493 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef8c80-1fd8-4d64-a140-bbcc5f47c7c1 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:11.569738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:11.578820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaef8c80-1fd8-4d64-a140-bbcc5f47c7c1 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:11.583491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bfcf0b-4807-4eaa-bb58-f42cf2ef798e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:11.584081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfcf0b-4807-4eaa-bb58-f42cf2ef798e admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:11.584363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfcf0b-4807-4eaa-bb58-f42cf2ef798e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:11.584634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bfcf0b-4807-4eaa-bb58-f42cf2ef798e admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:11.585310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bfcf0b-4807-4eaa-bb58-f42cf2ef798e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:11.587244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bfcf0b-4807-4eaa-bb58-f42cf2ef798e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:11.626386 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 793/1586] 10.222.0.16 () {58 vars in 1058 bytes} [Fri Jun 24 04:17:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:11.627321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e922054-0bc9-4ff4-90e4-e5e458496973 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:11.637049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e922054-0bc9-4ff4-90e4-e5e458496973 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:11.642169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0190c-ba38-4c86-adec-fcc1df8c68fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:11.647732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0190c-ba38-4c86-adec-fcc1df8c68fc admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:11.647732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0190c-ba38-4c86-adec-fcc1df8c68fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:11.647732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0190c-ba38-4c86-adec-fcc1df8c68fc admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:11.647732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0190c-ba38-4c86-adec-fcc1df8c68fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:11.647732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0190c-ba38-4c86-adec-fcc1df8c68fc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:11.696247 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 794/1587] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:17:11 2022] POST /identity/v3/users => generated 288 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:11.697182 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bfcf0b-4807-4eaa-bb58-f42cf2ef798e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:11.706827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bfcf0b-4807-4eaa-bb58-f42cf2ef798e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:11.711304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f14e6a-d185-45a7-a61c-4f2c7a39d3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:11.716676 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f14e6a-d185-45a7-a61c-4f2c7a39d3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:11.716676 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f14e6a-d185-45a7-a61c-4f2c7a39d3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:11.716676 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f14e6a-d185-45a7-a61c-4f2c7a39d3a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:11.716676 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f14e6a-d185-45a7-a61c-4f2c7a39d3a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:11.716676 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f14e6a-d185-45a7-a61c-4f2c7a39d3a8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:11.718921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 794/1588] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:17:11 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:11.722082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0190c-ba38-4c86-adec-fcc1df8c68fc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:11.729541 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0190c-ba38-4c86-adec-fcc1df8c68fc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:11.734214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201e16c-30fe-4d29-b303-d0ec8dba9ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:11.735124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201e16c-30fe-4d29-b303-d0ec8dba9ac2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:11.735411 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201e16c-30fe-4d29-b303-d0ec8dba9ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:11.735690 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201e16c-30fe-4d29-b303-d0ec8dba9ac2 admin admin] PATH_INFO: `/v3/projects/d56eee171a5c4f728bec99eef1f3b0a3/users/9ffa029a3b1e4919b00e58a9737816d2/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:11.750672 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201e16c-30fe-4d29-b303-d0ec8dba9ac2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56eee171a5c4f728bec99eef1f3b0a3, user_id=9ffa029a3b1e4919b00e58a9737816d2, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:11.753385 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201e16c-30fe-4d29-b303-d0ec8dba9ac2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:11.786724 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 795/1589] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:17:11 2022] POST /identity/v3/users => generated 284 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:11.787552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f14e6a-d185-45a7-a61c-4f2c7a39d3a8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:11.796945 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f14e6a-d185-45a7-a61c-4f2c7a39d3a8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:11.801408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24db744-aa12-4859-8256-6acdbb65f84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:11.802038 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24db744-aa12-4859-8256-6acdbb65f84f admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:11.802400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24db744-aa12-4859-8256-6acdbb65f84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:11.802691 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24db744-aa12-4859-8256-6acdbb65f84f admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:11.803401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24db744-aa12-4859-8256-6acdbb65f84f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:11.804985 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24db744-aa12-4859-8256-6acdbb65f84f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:11.810076 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 795/1590] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:11 2022] PUT /identity/v3/projects/d56eee171a5c4f728bec99eef1f3b0a3/users/9ffa029a3b1e4919b00e58a9737816d2/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:11.811121 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201e16c-30fe-4d29-b303-d0ec8dba9ac2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:11.819082 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 796/1591] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:11.819082 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24db744-aa12-4859-8256-6acdbb65f84f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:11.827004 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24db744-aa12-4859-8256-6acdbb65f84f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:11.877392 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201e16c-30fe-4d29-b303-d0ec8dba9ac2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:11.882703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b00c0-a879-4958-8378-0e92f69eb361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:11.883518 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d8d1f-b506-4899-b43e-d2caf7aeea8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:11.883650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b00c0-a879-4958-8378-0e92f69eb361 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:11.883937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b00c0-a879-4958-8378-0e92f69eb361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:11.884275 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b00c0-a879-4958-8378-0e92f69eb361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:11.884403 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d8d1f-b506-4899-b43e-d2caf7aeea8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:11.884685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d8d1f-b506-4899-b43e-d2caf7aeea8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:11.884971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d8d1f-b506-4899-b43e-d2caf7aeea8f admin admin] PATH_INFO: `/v3/projects/ab7130ecbc4d487db66baebeab8550b0/users/114d227e9645409e9082cf9b09b05c3b/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:11.885311 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b00c0-a879-4958-8378-0e92f69eb361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:11.887337 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1b00c0-a879-4958-8378-0e92f69eb361 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:11.898493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 796/1592] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:11.899396 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b00c0-a879-4958-8378-0e92f69eb361 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:11.900595 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d8d1f-b506-4899-b43e-d2caf7aeea8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab7130ecbc4d487db66baebeab8550b0, user_id=114d227e9645409e9082cf9b09b05c3b, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:11.903719 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d8d1f-b506-4899-b43e-d2caf7aeea8f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:11.955455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:11.968579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1b00c0-a879-4958-8378-0e92f69eb361 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:11.970310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 797/1593] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:11 2022] PUT /identity/v3/projects/ab7130ecbc4d487db66baebeab8550b0/users/114d227e9645409e9082cf9b09b05c3b/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:11.971156 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d8d1f-b506-4899-b43e-d2caf7aeea8f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:12.032004 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d8d1f-b506-4899-b43e-d2caf7aeea8f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:12.036632 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee5651-090c-441d-9aed-dc083052dd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:12.037310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee5651-090c-441d-9aed-dc083052dd47 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.037596 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee5651-090c-441d-9aed-dc083052dd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.037873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee5651-090c-441d-9aed-dc083052dd47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.038637 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee5651-090c-441d-9aed-dc083052dd47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:12.040247 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee5651-090c-441d-9aed-dc083052dd47 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:12.042033 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70950e-cf1f-46dd-8889-000e070957e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:12.042732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70950e-cf1f-46dd-8889-000e070957e3 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.043020 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70950e-cf1f-46dd-8889-000e070957e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.043708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70950e-cf1f-46dd-8889-000e070957e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.051025 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 798/1594] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:12.051849 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee5651-090c-441d-9aed-dc083052dd47 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:12.061317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee5651-090c-441d-9aed-dc083052dd47 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:12.066894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336d3d4-41c9-4b9b-88d8-f9ffcaaa7d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:12.067538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336d3d4-41c9-4b9b-88d8-f9ffcaaa7d7b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.067824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336d3d4-41c9-4b9b-88d8-f9ffcaaa7d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.068103 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336d3d4-41c9-4b9b-88d8-f9ffcaaa7d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.068819 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336d3d4-41c9-4b9b-88d8-f9ffcaaa7d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:12.070422 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336d3d4-41c9-4b9b-88d8-f9ffcaaa7d7b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:12.080482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 799/1595] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:12.081096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336d3d4-41c9-4b9b-88d8-f9ffcaaa7d7b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:12.086878 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:12.090580 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336d3d4-41c9-4b9b-88d8-f9ffcaaa7d7b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:12.095459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fd9eb2-6329-4a0b-9360-6dd4109de5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:12.097810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd9eb2-6329-4a0b-9360-6dd4109de5dc admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.097810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd9eb2-6329-4a0b-9360-6dd4109de5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.097810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fd9eb2-6329-4a0b-9360-6dd4109de5dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.097810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd9eb2-6329-4a0b-9360-6dd4109de5dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:12.098920 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fd9eb2-6329-4a0b-9360-6dd4109de5dc admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:12.101323 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70950e-cf1f-46dd-8889-000e070957e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:12.102946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70950e-cf1f-46dd-8889-000e070957e3 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:12.110101 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 800/1596] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:12.111164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb105dd2-0ff8-4756-b428-da009439d4d3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.111465 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb105dd2-0ff8-4756-b428-da009439d4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.111720 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb105dd2-0ff8-4756-b428-da009439d4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.112867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 797/1597] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:12.113703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70950e-cf1f-46dd-8889-000e070957e3 service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:12.123675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70950e-cf1f-46dd-8889-000e070957e3 service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:12.129529 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c60951-9375-43b0-95f5-34577efa4987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:12.130131 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c60951-9375-43b0-95f5-34577efa4987 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.130443 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c60951-9375-43b0-95f5-34577efa4987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.130744 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-eb105dd2-0ff8-4756-b428-da009439d4d3 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:12.130853 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c60951-9375-43b0-95f5-34577efa4987 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.131443 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c60951-9375-43b0-95f5-34577efa4987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:12.131942 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-eb105dd2-0ff8-4756-b428-da009439d4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:12.133068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c60951-9375-43b0-95f5-34577efa4987 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:12.140132 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 801/1598] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:12.147435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 798/1599] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:12.147544 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb105dd2-0ff8-4756-b428-da009439d4d3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:12.147636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5046dba-b018-4aea-a4d3-b8a16d04c9eb None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.147636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5046dba-b018-4aea-a4d3-b8a16d04c9eb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.147636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5046dba-b018-4aea-a4d3-b8a16d04c9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.154292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb105dd2-0ff8-4756-b428-da009439d4d3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:12.159219 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ea185-7732-4aa9-9f7d-a359c2dec2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:12.159905 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ea185-7732-4aa9-9f7d-a359c2dec2bd admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.160997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ea185-7732-4aa9-9f7d-a359c2dec2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.161671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ea185-7732-4aa9-9f7d-a359c2dec2bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.162478 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ea185-7732-4aa9-9f7d-a359c2dec2bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:12.164563 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ea185-7732-4aa9-9f7d-a359c2dec2bd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:12.165857 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b5046dba-b018-4aea-a4d3-b8a16d04c9eb None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:12.167103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b5046dba-b018-4aea-a4d3-b8a16d04c9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:12.178832 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 799/1600] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:12.178832 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5046dba-b018-4aea-a4d3-b8a16d04c9eb None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:12.183840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 802/1601] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:12.184726 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ea185-7732-4aa9-9f7d-a359c2dec2bd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:12.188617 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5046dba-b018-4aea-a4d3-b8a16d04c9eb None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:12.194846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a879da7-9499-4cdb-b959-bf27311a3021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:12.195513 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a879da7-9499-4cdb-b959-bf27311a3021 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.195792 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a879da7-9499-4cdb-b959-bf27311a3021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.196076 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a879da7-9499-4cdb-b959-bf27311a3021 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.196803 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a879da7-9499-4cdb-b959-bf27311a3021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:12.198240 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ea185-7732-4aa9-9f7d-a359c2dec2bd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:12.198459 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a879da7-9499-4cdb-b959-bf27311a3021 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:12.203412 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f8574-9a97-4ba8-9d79-b8e0ae8ed241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:12.204245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f8574-9a97-4ba8-9d79-b8e0ae8ed241 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.204482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f8574-9a97-4ba8-9d79-b8e0ae8ed241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.204953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f8574-9a97-4ba8-9d79-b8e0ae8ed241 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.205761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f8574-9a97-4ba8-9d79-b8e0ae8ed241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:12.207432 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f8574-9a97-4ba8-9d79-b8e0ae8ed241 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:12.212122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 800/1602] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:12.212122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a879da7-9499-4cdb-b959-bf27311a3021 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:12.219732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a879da7-9499-4cdb-b959-bf27311a3021 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:12.224704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 803/1603] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:12.224704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f8574-9a97-4ba8-9d79-b8e0ae8ed241 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:12.227535 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad9d50-dee4-494e-8496-b57991fa7b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:12.228266 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad9d50-dee4-494e-8496-b57991fa7b75 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.228619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad9d50-dee4-494e-8496-b57991fa7b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.228968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad9d50-dee4-494e-8496-b57991fa7b75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.229779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad9d50-dee4-494e-8496-b57991fa7b75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:12.231416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad9d50-dee4-494e-8496-b57991fa7b75 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:12.235915 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f8574-9a97-4ba8-9d79-b8e0ae8ed241 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:12.240891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990b4f7-662e-444b-be22-fac6c8f31818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:12.241769 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990b4f7-662e-444b-be22-fac6c8f31818 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.242053 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990b4f7-662e-444b-be22-fac6c8f31818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.242331 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990b4f7-662e-444b-be22-fac6c8f31818 admin admin] PATH_INFO: `/v3/projects/a48b90744f124a21b9e8d2b78836462e/users/aa8237a791034b80973d5cc4329d015c/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.245987 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 801/1604] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:12.245987 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad9d50-dee4-494e-8496-b57991fa7b75 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:12.261795 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990b4f7-662e-444b-be22-fac6c8f31818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a48b90744f124a21b9e8d2b78836462e, user_id=aa8237a791034b80973d5cc4329d015c, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:12.264663 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2990b4f7-662e-444b-be22-fac6c8f31818 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:12.300584 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad9d50-dee4-494e-8496-b57991fa7b75 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:12.304237 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558553f-3e42-4c95-90e8-cd9c1a9555a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:12.304880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558553f-3e42-4c95-90e8-cd9c1a9555a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.305282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558553f-3e42-4c95-90e8-cd9c1a9555a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.305565 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c558553f-3e42-4c95-90e8-cd9c1a9555a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.315334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558553f-3e42-4c95-90e8-cd9c1a9555a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:12.317841 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c558553f-3e42-4c95-90e8-cd9c1a9555a1 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:12.321737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:12.326665 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 802/1605] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:12.327474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558553f-3e42-4c95-90e8-cd9c1a9555a1 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:12.344034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c558553f-3e42-4c95-90e8-cd9c1a9555a1 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:12.348094 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 804/1606] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:12 2022] PUT /identity/v3/projects/a48b90744f124a21b9e8d2b78836462e/users/aa8237a791034b80973d5cc4329d015c/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:12.349042 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990b4f7-662e-444b-be22-fac6c8f31818 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:12.408820 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ef934-18e0-4027-9730-3f3f42d993cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:12.409813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ef934-18e0-4027-9730-3f3f42d993cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.410174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ef934-18e0-4027-9730-3f3f42d993cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.410563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ef934-18e0-4027-9730-3f3f42d993cd admin admin] PATH_INFO: `/v3/projects/570a83d81ade40c08d677db7c3ad102f/users/7ab0e61b138e43d39bc2965354195c0f/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.424086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990b4f7-662e-444b-be22-fac6c8f31818 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:12.427388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936ef934-18e0-4027-9730-3f3f42d993cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=570a83d81ade40c08d677db7c3ad102f, user_id=7ab0e61b138e43d39bc2965354195c0f, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:12.428601 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee4e09-58cf-4dc5-b2cd-a4f03b51e3a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:12.429260 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee4e09-58cf-4dc5-b2cd-a4f03b51e3a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.429546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee4e09-58cf-4dc5-b2cd-a4f03b51e3a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.429821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee4e09-58cf-4dc5-b2cd-a4f03b51e3a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.431197 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936ef934-18e0-4027-9730-3f3f42d993cd admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:12.442395 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee4e09-58cf-4dc5-b2cd-a4f03b51e3a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:12.443959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee4e09-58cf-4dc5-b2cd-a4f03b51e3a8 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:12.455978 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 805/1607] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:12.457016 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee4e09-58cf-4dc5-b2cd-a4f03b51e3a8 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:12.467501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee4e09-58cf-4dc5-b2cd-a4f03b51e3a8 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:12.473538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6eb616-2582-49bf-bce9-e6ba35871adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:12.474513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6eb616-2582-49bf-bce9-e6ba35871adc admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.474800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6eb616-2582-49bf-bce9-e6ba35871adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.475076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6eb616-2582-49bf-bce9-e6ba35871adc admin admin] PATH_INFO: `/v3/projects/ba92bd9c1afc47008582aba7ec0c5266/users/066d1521db464b4996f6daf6548dcc38/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.492728 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6eb616-2582-49bf-bce9-e6ba35871adc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba92bd9c1afc47008582aba7ec0c5266, user_id=066d1521db464b4996f6daf6548dcc38, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:12.495597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6eb616-2582-49bf-bce9-e6ba35871adc admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:12.557115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 803/1608] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:12 2022] PUT /identity/v3/projects/570a83d81ade40c08d677db7c3ad102f/users/7ab0e61b138e43d39bc2965354195c0f/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:12.557115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ef934-18e0-4027-9730-3f3f42d993cd admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:12.602808 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 806/1609] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:12 2022] PUT /identity/v3/projects/ba92bd9c1afc47008582aba7ec0c5266/users/066d1521db464b4996f6daf6548dcc38/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:12.602808 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6eb616-2582-49bf-bce9-e6ba35871adc admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:12.654908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ef934-18e0-4027-9730-3f3f42d993cd admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:12.717604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6eb616-2582-49bf-bce9-e6ba35871adc admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:12.728226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea386b-511c-4268-8db8-07731f50b8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:12.728226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea386b-511c-4268-8db8-07731f50b8e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.728226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea386b-511c-4268-8db8-07731f50b8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.728226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea386b-511c-4268-8db8-07731f50b8e4 admin admin] PATH_INFO: `/v3/projects/3df9eb441b5440f6800e5ce6ae3395b3/users/50be5e96476a4a8fb6acb125d3f99841/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.752019 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea386b-511c-4268-8db8-07731f50b8e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df9eb441b5440f6800e5ce6ae3395b3, user_id=50be5e96476a4a8fb6acb125d3f99841, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:12.755004 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea386b-511c-4268-8db8-07731f50b8e4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:12.755294 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727673ba-3b00-4733-8f5c-8136cac3bf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:12.756207 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727673ba-3b00-4733-8f5c-8136cac3bf7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.756491 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727673ba-3b00-4733-8f5c-8136cac3bf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.756768 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727673ba-3b00-4733-8f5c-8136cac3bf7f admin admin] PATH_INFO: `/v3/projects/96e2b5a124bf48fd82e48ed79aa4eeed/users/b158078d039a4979a9c0cf2be26c4b14/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.773384 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727673ba-3b00-4733-8f5c-8136cac3bf7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=96e2b5a124bf48fd82e48ed79aa4eeed, user_id=b158078d039a4979a9c0cf2be26c4b14, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:12.776280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727673ba-3b00-4733-8f5c-8136cac3bf7f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:12.835473 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 807/1610] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:12 2022] PUT /identity/v3/projects/3df9eb441b5440f6800e5ce6ae3395b3/users/50be5e96476a4a8fb6acb125d3f99841/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:12.836788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85306dc-b02a-4e89-a752-9108493ffc99 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.837080 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85306dc-b02a-4e89-a752-9108493ffc99 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.837360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85306dc-b02a-4e89-a752-9108493ffc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.843076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:12.856004 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d85306dc-b02a-4e89-a752-9108493ffc99 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:12.863661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d85306dc-b02a-4e89-a752-9108493ffc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:12.899062 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 804/1611] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:12 2022] PUT /identity/v3/projects/96e2b5a124bf48fd82e48ed79aa4eeed/users/b158078d039a4979a9c0cf2be26c4b14/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:12.899062 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f2c7f-34ef-4970-8b57-823254f3705f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.899062 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f2c7f-34ef-4970-8b57-823254f3705f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.899062 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f2c7f-34ef-4970-8b57-823254f3705f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.934591 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:12.939600 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a52f2c7f-34ef-4970-8b57-823254f3705f None None] MFA Rules not processed for user `7ab0e61b138e43d39bc2965354195c0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:12.941293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 808/1612] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:12.941439 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a52f2c7f-34ef-4970-8b57-823254f3705f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:12.942915 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a79016-24eb-4e38-b4b0-14c11e23181c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:12.943384 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a79016-24eb-4e38-b4b0-14c11e23181c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:12.943808 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a79016-24eb-4e38-b4b0-14c11e23181c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:12.962711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b1a79016-24eb-4e38-b4b0-14c11e23181c None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:12.964061 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b1a79016-24eb-4e38-b4b0-14c11e23181c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:13.019575 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 805/1613] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:12 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:13.027767 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cb3ef-6376-4542-8ea9-96b14c0fa61e None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:13.027991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cb3ef-6376-4542-8ea9-96b14c0fa61e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:13.028340 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cb3ef-6376-4542-8ea9-96b14c0fa61e None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:13.030434 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 806/1614] 10.222.0.16 () {54 vars in 961 bytes} [Fri Jun 24 04:17:13 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:17:13.036361 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef340f-184c-4fa0-9c02-8aa5e62506df None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:13.036361 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef340f-184c-4fa0-9c02-8aa5e62506df None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:13.036361 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef340f-184c-4fa0-9c02-8aa5e62506df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:13.072832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 809/1615] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:13.075335 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486731dd-ed91-4256-b7c8-621f50cfac0f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:13.075633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486731dd-ed91-4256-b7c8-621f50cfac0f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:13.075914 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486731dd-ed91-4256-b7c8-621f50cfac0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:13.080088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f4ef340f-184c-4fa0-9c02-8aa5e62506df None None] MFA Rules not processed for user `50be5e96476a4a8fb6acb125d3f99841`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:13.081824 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f4ef340f-184c-4fa0-9c02-8aa5e62506df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:13.097557 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-486731dd-ed91-4256-b7c8-621f50cfac0f None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:13.098943 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-486731dd-ed91-4256-b7c8-621f50cfac0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:13.107867 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 810/1616] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:13.108939 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486731dd-ed91-4256-b7c8-621f50cfac0f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:13.181804 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 807/1617] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:13 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:13.182976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef340f-184c-4fa0-9c02-8aa5e62506df None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:13.196019 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486731dd-ed91-4256-b7c8-621f50cfac0f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:13.196598 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef340f-184c-4fa0-9c02-8aa5e62506df None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:13.200104 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4f4f7-c2e2-4097-ae86-6454fbdda81c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:13.201059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4f4f7-c2e2-4097-ae86-6454fbdda81c service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:13.202140 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a85af0-4059-4a34-9ad8-03677ffc8661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:13.202394 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4f4f7-c2e2-4097-ae86-6454fbdda81c service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:13.202820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4f4f7-c2e2-4097-ae86-6454fbdda81c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:13.202960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a85af0-4059-4a34-9ad8-03677ffc8661 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:13.203211 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a85af0-4059-4a34-9ad8-03677ffc8661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:13.203505 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a85af0-4059-4a34-9ad8-03677ffc8661 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:13.205143 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a85af0-4059-4a34-9ad8-03677ffc8661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:13.207365 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a85af0-4059-4a34-9ad8-03677ffc8661 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:13.214517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f4f4f7-c2e2-4097-ae86-6454fbdda81c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:13.216180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f4f4f7-c2e2-4097-ae86-6454fbdda81c service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:13.231424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 811/1618] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:13.232790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfcccf-334d-4a24-bd33-c91823b66492 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:13.233087 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfcccf-334d-4a24-bd33-c91823b66492 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:13.233363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abfcccf-334d-4a24-bd33-c91823b66492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:13.253278 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7abfcccf-334d-4a24-bd33-c91823b66492 None None] MFA Rules not processed for user `80b7b3ec60db4de19bbf7d08f111f470`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:13.254529 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7abfcccf-334d-4a24-bd33-c91823b66492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:13.290453 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:13.314166 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 808/1619] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:17:13 2022] POST /identity/v3/projects => generated 363 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:13.315114 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a85af0-4059-4a34-9ad8-03677ffc8661 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:13.332351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 812/1620] 10.222.0.16 () {58 vars in 1058 bytes} [Fri Jun 24 04:17:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:13.336361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfcccf-334d-4a24-bd33-c91823b66492 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:13.347271 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abfcccf-334d-4a24-bd33-c91823b66492 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:13.412185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a85af0-4059-4a34-9ad8-03677ffc8661 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:13.418634 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd1c42-e75f-470f-9119-7958ab436aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:13.418820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922710cb-1627-4599-89d1-7501d5b3e3c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:13.419304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd1c42-e75f-470f-9119-7958ab436aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:13.419538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922710cb-1627-4599-89d1-7501d5b3e3c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:13.419664 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd1c42-e75f-470f-9119-7958ab436aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:13.419948 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdd1c42-e75f-470f-9119-7958ab436aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:13.420214 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922710cb-1627-4599-89d1-7501d5b3e3c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:13.420759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922710cb-1627-4599-89d1-7501d5b3e3c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:13.456608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:13.496581 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd1c42-e75f-470f-9119-7958ab436aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:13.498149 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdd1c42-e75f-470f-9119-7958ab436aa0 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:13.503683 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922710cb-1627-4599-89d1-7501d5b3e3c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:13.505145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922710cb-1627-4599-89d1-7501d5b3e3c9 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:13.510236 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 809/1621] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:13.511405 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd1c42-e75f-470f-9119-7958ab436aa0 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:13.518069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 813/1622] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:13.519139 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922710cb-1627-4599-89d1-7501d5b3e3c9 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:13.529560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922710cb-1627-4599-89d1-7501d5b3e3c9 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:13.537011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ddffd-3dc0-40f8-9516-bc2bbca980d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:13.538710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ddffd-3dc0-40f8-9516-bc2bbca980d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:13.538710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ddffd-3dc0-40f8-9516-bc2bbca980d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:13.539097 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69ddffd-3dc0-40f8-9516-bc2bbca980d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:13.539806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ddffd-3dc0-40f8-9516-bc2bbca980d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:13.542491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69ddffd-3dc0-40f8-9516-bc2bbca980d4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:13.576584 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdd1c42-e75f-470f-9119-7958ab436aa0 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:13.580903 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c44e1-4f7b-4d97-be09-e219125d572c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:13.581591 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c44e1-4f7b-4d97-be09-e219125d572c service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:13.581876 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c44e1-4f7b-4d97-be09-e219125d572c service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:13.582151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c44e1-4f7b-4d97-be09-e219125d572c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:13.666279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c44e1-4f7b-4d97-be09-e219125d572c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:13.667774 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c44e1-4f7b-4d97-be09-e219125d572c service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:13.679070 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 810/1623] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:13 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:13.680428 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6900a59-f616-4330-bd86-7baea67cb9f3 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:13.680721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6900a59-f616-4330-bd86-7baea67cb9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:13.681001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6900a59-f616-4330-bd86-7baea67cb9f3 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:13.682452 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 811/1624] 10.222.0.16 () {54 vars in 912 bytes} [Fri Jun 24 04:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:17:13.688473 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622447ff-a645-4408-aa59-ef7309335198 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:13.688848 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622447ff-a645-4408-aa59-ef7309335198 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:13.690657 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622447ff-a645-4408-aa59-ef7309335198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:13.714536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-622447ff-a645-4408-aa59-ef7309335198 None None] MFA Rules not processed for user `80b7b3ec60db4de19bbf7d08f111f470`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:13.716038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-622447ff-a645-4408-aa59-ef7309335198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:13.734799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 814/1625] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:17:13 2022] POST /identity/v3/users => generated 288 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:13.739519 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ddffd-3dc0-40f8-9516-bc2bbca980d4 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:13.763464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69ddffd-3dc0-40f8-9516-bc2bbca980d4 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:13.773656 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0e6a9-61e3-4bca-b34a-14ffbeb02767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:13.776201 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0e6a9-61e3-4bca-b34a-14ffbeb02767 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:13.777511 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0e6a9-61e3-4bca-b34a-14ffbeb02767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:13.777877 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0e6a9-61e3-4bca-b34a-14ffbeb02767 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:13.780623 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0e6a9-61e3-4bca-b34a-14ffbeb02767 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:13.783692 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b0e6a9-61e3-4bca-b34a-14ffbeb02767 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:13.807069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 815/1626] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:13.809172 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0e6a9-61e3-4bca-b34a-14ffbeb02767 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:13.826124 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b0e6a9-61e3-4bca-b34a-14ffbeb02767 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:13.836213 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179972ff-512b-4694-bb23-762da4346d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:13.837373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179972ff-512b-4694-bb23-762da4346d97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:13.837852 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179972ff-512b-4694-bb23-762da4346d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:13.838616 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179972ff-512b-4694-bb23-762da4346d97 admin admin] PATH_INFO: `/v3/projects/2e1e3c6f45844e038e95f047768a0fff/users/255d98b204724decac5de6d284b9f2ce/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:13.847074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 812/1627] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jun 24 04:17:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:17:13.855865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622447ff-a645-4408-aa59-ef7309335198 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:13.868823 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622447ff-a645-4408-aa59-ef7309335198 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:13.871684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179972ff-512b-4694-bb23-762da4346d97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e1e3c6f45844e038e95f047768a0fff, user_id=255d98b204724decac5de6d284b9f2ce, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:13.876341 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179972ff-512b-4694-bb23-762da4346d97 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:13.883179 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65fef6-29f5-4e5d-97e1-e3e01dc7e27b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:13.884148 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65fef6-29f5-4e5d-97e1-e3e01dc7e27b service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:13.884541 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65fef6-29f5-4e5d-97e1-e3e01dc7e27b service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:13.885172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65fef6-29f5-4e5d-97e1-e3e01dc7e27b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:13.901823 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65fef6-29f5-4e5d-97e1-e3e01dc7e27b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:13.904878 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65fef6-29f5-4e5d-97e1-e3e01dc7e27b service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:13.938199 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 813/1628] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:13.962978 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 816/1629] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:13 2022] PUT /identity/v3/projects/2e1e3c6f45844e038e95f047768a0fff/users/255d98b204724decac5de6d284b9f2ce/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:13.966647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65fef6-29f5-4e5d-97e1-e3e01dc7e27b service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:13.983471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65fef6-29f5-4e5d-97e1-e3e01dc7e27b service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:14.010518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:14.126447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ee153-4939-480b-9d4e-54c7f2555175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:14.127149 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ee153-4939-480b-9d4e-54c7f2555175 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:14.127472 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ee153-4939-480b-9d4e-54c7f2555175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:14.127759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ee153-4939-480b-9d4e-54c7f2555175 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:14.129162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ee153-4939-480b-9d4e-54c7f2555175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:14.132563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ee153-4939-480b-9d4e-54c7f2555175 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:14.138971 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6281f-94dd-4a9f-9ea8-2122b92734cd None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:14.139332 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6281f-94dd-4a9f-9ea8-2122b92734cd None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:14.139671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6281f-94dd-4a9f-9ea8-2122b92734cd None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:14.142729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 817/1630] 10.222.0.16 () {54 vars in 912 bytes} [Fri Jun 24 04:17:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:17:14.148505 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2618c5d-83e4-44fd-b782-a363235a2728 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:14.148794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2618c5d-83e4-44fd-b782-a363235a2728 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:14.149088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2618c5d-83e4-44fd-b782-a363235a2728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:14.175668 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 814/1631] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:13 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:14.177824 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ee153-4939-480b-9d4e-54c7f2555175 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:14.198647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ee153-4939-480b-9d4e-54c7f2555175 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:14.201157 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e2618c5d-83e4-44fd-b782-a363235a2728 None None] MFA Rules not processed for user `80b7b3ec60db4de19bbf7d08f111f470`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:14.204212 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e2618c5d-83e4-44fd-b782-a363235a2728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:14.214045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd0af40-b604-4ece-8346-e0b202330250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:14.215143 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0af40-b604-4ece-8346-e0b202330250 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:14.215787 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0af40-b604-4ece-8346-e0b202330250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:14.216173 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd0af40-b604-4ece-8346-e0b202330250 admin admin] PATH_INFO: `/v3/projects/2e1e3c6f45844e038e95f047768a0fff/users/255d98b204724decac5de6d284b9f2ce/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:14.221020 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0af40-b604-4ece-8346-e0b202330250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e1e3c6f45844e038e95f047768a0fff, user_id=255d98b204724decac5de6d284b9f2ce, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:14.223985 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd0af40-b604-4ece-8346-e0b202330250 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:14.251165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:14.252630 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 815/1632] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:14 2022] PUT /identity/v3/projects/2e1e3c6f45844e038e95f047768a0fff/users/255d98b204724decac5de6d284b9f2ce/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:14.263896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773bf4c-bef8-46fd-afbd-59c091c2338d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:14.264262 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773bf4c-bef8-46fd-afbd-59c091c2338d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:14.265477 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773bf4c-bef8-46fd-afbd-59c091c2338d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:14.325398 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4773bf4c-bef8-46fd-afbd-59c091c2338d None None] MFA Rules not processed for user `255d98b204724decac5de6d284b9f2ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:14.326981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4773bf4c-bef8-46fd-afbd-59c091c2338d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:14.355563 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 818/1633] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jun 24 04:17:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:17:14.369650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2618c5d-83e4-44fd-b782-a363235a2728 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:14.423634 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 816/1634] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:14 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:14.462557 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2618c5d-83e4-44fd-b782-a363235a2728 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:14.524035 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046a7ae-027f-4f65-a21f-04d51acf527c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:14.524937 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046a7ae-027f-4f65-a21f-04d51acf527c service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:14.525322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046a7ae-027f-4f65-a21f-04d51acf527c service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:14.526242 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046a7ae-027f-4f65-a21f-04d51acf527c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:14.632668 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046a7ae-027f-4f65-a21f-04d51acf527c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:14.634320 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046a7ae-027f-4f65-a21f-04d51acf527c service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:14.654608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 819/1635] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:14.698955 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773bf4c-bef8-46fd-afbd-59c091c2338d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:14.742115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773bf4c-bef8-46fd-afbd-59c091c2338d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:14.756093 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635717bd-fb78-4eb7-8e24-79c85727c431 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:14.756093 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635717bd-fb78-4eb7-8e24-79c85727c431 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:14.756093 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635717bd-fb78-4eb7-8e24-79c85727c431 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:14.756093 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635717bd-fb78-4eb7-8e24-79c85727c431 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:14.766458 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635717bd-fb78-4eb7-8e24-79c85727c431 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:14.768089 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635717bd-fb78-4eb7-8e24-79c85727c431 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:14.780946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 817/1636] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:14 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:14.974114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046a7ae-027f-4f65-a21f-04d51acf527c service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:15.164118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046a7ae-027f-4f65-a21f-04d51acf527c service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:15.172684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ada39-9f4e-4cf3-9f4c-80f6681dc979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:15.174018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ada39-9f4e-4cf3-9f4c-80f6681dc979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:15.174327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ada39-9f4e-4cf3-9f4c-80f6681dc979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:15.174649 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ada39-9f4e-4cf3-9f4c-80f6681dc979 admin admin] PATH_INFO: `/v3/users/b938baf805604bd9ab55d1c364daf426` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:15.176011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ada39-9f4e-4cf3-9f4c-80f6681dc979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b938baf805604bd9ab55d1c364daf426)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:15.177402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: WARNING py.warnings [None req-963ada39-9f4e-4cf3-9f4c-80f6681dc979 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 04:17:15.177402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: warnings.warn(msg) Jun 24 04:17:15.177402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: Jun 24 04:17:15.180273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ada39-9f4e-4cf3-9f4c-80f6681dc979 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:15.485964 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-963ada39-9f4e-4cf3-9f4c-80f6681dc979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b938baf805604bd9ab55d1c364daf426'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:15.514025 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-963ada39-9f4e-4cf3-9f4c-80f6681dc979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b938baf805604bd9ab55d1c364daf426'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:15.515512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-963ada39-9f4e-4cf3-9f4c-80f6681dc979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b938baf805604bd9ab55d1c364daf426'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:15.516214 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-963ada39-9f4e-4cf3-9f4c-80f6681dc979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b938baf805604bd9ab55d1c364daf426'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:15.545025 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-963ada39-9f4e-4cf3-9f4c-80f6681dc979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b938baf805604bd9ab55d1c364daf426'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:15.599127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:15.681524 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 820/1637] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:17:14 2022] DELETE /identity/v3/users/b938baf805604bd9ab55d1c364daf426 => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:15.687151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a5edf-a2cd-4c13-84d5-750c42505ace None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:15.687426 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a5edf-a2cd-4c13-84d5-750c42505ace None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:15.687715 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a5edf-a2cd-4c13-84d5-750c42505ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:15.725253 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-822a5edf-a2cd-4c13-84d5-750c42505ace None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:15.727257 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-822a5edf-a2cd-4c13-84d5-750c42505ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:15.778054 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:15.794237 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 818/1638] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:15.804234 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ada39-9f4e-4cf3-9f4c-80f6681dc979 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:15.805941 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-963ada39-9f4e-4cf3-9f4c-80f6681dc979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:15.966543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ada39-9f4e-4cf3-9f4c-80f6681dc979 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:15.971491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940934a7-e1d0-48c3-b5c0-3c3d4d0805e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:15.972238 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940934a7-e1d0-48c3-b5c0-3c3d4d0805e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:15.972624 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940934a7-e1d0-48c3-b5c0-3c3d4d0805e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:15.973039 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940934a7-e1d0-48c3-b5c0-3c3d4d0805e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:16.252271 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940934a7-e1d0-48c3-b5c0-3c3d4d0805e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:16.252271 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940934a7-e1d0-48c3-b5c0-3c3d4d0805e0 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:16.252271 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 821/1639] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:17:16.473726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a5edf-a2cd-4c13-84d5-750c42505ace None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:16.477723 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-822a5edf-a2cd-4c13-84d5-750c42505ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:16.493284 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a5edf-a2cd-4c13-84d5-750c42505ace None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:16.498760 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766db7c3-1a29-4545-97c9-9e15bf09ad1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:16.505868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766db7c3-1a29-4545-97c9-9e15bf09ad1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:16.505868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766db7c3-1a29-4545-97c9-9e15bf09ad1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:16.505868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766db7c3-1a29-4545-97c9-9e15bf09ad1b admin admin] PATH_INFO: `/v3/users/7ab0e61b138e43d39bc2965354195c0f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:16.505868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766db7c3-1a29-4545-97c9-9e15bf09ad1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ab0e61b138e43d39bc2965354195c0f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:16.505868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766db7c3-1a29-4545-97c9-9e15bf09ad1b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:16.813269 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-766db7c3-1a29-4545-97c9-9e15bf09ad1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ab0e61b138e43d39bc2965354195c0f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:16.833393 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-766db7c3-1a29-4545-97c9-9e15bf09ad1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ab0e61b138e43d39bc2965354195c0f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:16.959721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-766db7c3-1a29-4545-97c9-9e15bf09ad1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ab0e61b138e43d39bc2965354195c0f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:16.960876 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-766db7c3-1a29-4545-97c9-9e15bf09ad1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ab0e61b138e43d39bc2965354195c0f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:16.961988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-766db7c3-1a29-4545-97c9-9e15bf09ad1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ab0e61b138e43d39bc2965354195c0f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:17.005407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 819/1640] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:17:16 2022] DELETE /identity/v3/users/7ab0e61b138e43d39bc2965354195c0f => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:17.005609 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7247e-0291-4302-bf81-c1f57f954583 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:17.005609 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7247e-0291-4302-bf81-c1f57f954583 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:17.005609 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7247e-0291-4302-bf81-c1f57f954583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:17.050925 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-54a7247e-0291-4302-bf81-c1f57f954583 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:17.055339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-54a7247e-0291-4302-bf81-c1f57f954583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:17.125344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:17.215461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 822/1641] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:17.224114 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766db7c3-1a29-4545-97c9-9e15bf09ad1b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:17.225460 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-766db7c3-1a29-4545-97c9-9e15bf09ad1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:17.242954 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7247e-0291-4302-bf81-c1f57f954583 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:17.245340 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-54a7247e-0291-4302-bf81-c1f57f954583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:17.262234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766db7c3-1a29-4545-97c9-9e15bf09ad1b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:17.281361 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76663d97-58ed-4a33-9777-220756a4b521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:17.290964 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76663d97-58ed-4a33-9777-220756a4b521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:17.291540 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76663d97-58ed-4a33-9777-220756a4b521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:17.292210 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76663d97-58ed-4a33-9777-220756a4b521 admin admin] PATH_INFO: `/v3/projects/cf6ac8a37c564019bdad3a570e9241f7` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:17.295471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76663d97-58ed-4a33-9777-220756a4b521 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf6ac8a37c564019bdad3a570e9241f7)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:17.302588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: WARNING py.warnings [None req-76663d97-58ed-4a33-9777-220756a4b521 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 04:17:17.302588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: warnings.warn(msg) Jun 24 04:17:17.302588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: Jun 24 04:17:17.304532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76663d97-58ed-4a33-9777-220756a4b521 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:17.405068 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7247e-0291-4302-bf81-c1f57f954583 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:17.409800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4aa2d-1aa0-4ca4-b053-cf176605ae40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:17.410876 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4aa2d-1aa0-4ca4-b053-cf176605ae40 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:17.411269 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4aa2d-1aa0-4ca4-b053-cf176605ae40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:17.411714 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4aa2d-1aa0-4ca4-b053-cf176605ae40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:17.435482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4aa2d-1aa0-4ca4-b053-cf176605ae40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:17.442214 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4aa2d-1aa0-4ca4-b053-cf176605ae40 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:17.460044 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 823/1642] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:17:17.523462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-76663d97-58ed-4a33-9777-220756a4b521 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf6ac8a37c564019bdad3a570e9241f7', 'request_id': 'req-76663d97-58ed-4a33-9777-220756a4b521'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:17.605957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-76663d97-58ed-4a33-9777-220756a4b521 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf6ac8a37c564019bdad3a570e9241f7', 'request_id': 'req-76663d97-58ed-4a33-9777-220756a4b521'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:17.613638 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-76663d97-58ed-4a33-9777-220756a4b521 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf6ac8a37c564019bdad3a570e9241f7', 'request_id': 'req-76663d97-58ed-4a33-9777-220756a4b521'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:17.668931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-76663d97-58ed-4a33-9777-220756a4b521 admin admin] The token cache is being invalidate because project cf6ac8a37c564019bdad3a570e9241f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:17:17.671673 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-76663d97-58ed-4a33-9777-220756a4b521 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:17.674078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-76663d97-58ed-4a33-9777-220756a4b521 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:17.676889 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 820/1643] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:17:17 2022] DELETE /identity/v3/projects/cf6ac8a37c564019bdad3a570e9241f7 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:17.681832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76991ca6-9a1f-4261-b520-fd51829cbbe6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:17.683998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76991ca6-9a1f-4261-b520-fd51829cbbe6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:17.684508 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76991ca6-9a1f-4261-b520-fd51829cbbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:17.741192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-76991ca6-9a1f-4261-b520-fd51829cbbe6 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:17.744085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-76991ca6-9a1f-4261-b520-fd51829cbbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:17.912535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 824/1644] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:17.920157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ae14b-7a10-407a-9d04-b406147fd9ca None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:17.920157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ae14b-7a10-407a-9d04-b406147fd9ca None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:17.920157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ae14b-7a10-407a-9d04-b406147fd9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:17.942675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-486ae14b-7a10-407a-9d04-b406147fd9ca None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:17.944152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-486ae14b-7a10-407a-9d04-b406147fd9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:17.961400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76991ca6-9a1f-4261-b520-fd51829cbbe6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:17.962146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 821/1645] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:17.976038 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76991ca6-9a1f-4261-b520-fd51829cbbe6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:17.982038 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a615bb-d648-4050-81ef-84866f66453d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:17.986571 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a615bb-d648-4050-81ef-84866f66453d admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:17.986571 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a615bb-d648-4050-81ef-84866f66453d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:17.986571 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a615bb-d648-4050-81ef-84866f66453d admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:17.986571 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a615bb-d648-4050-81ef-84866f66453d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:17:17.986571 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a615bb-d648-4050-81ef-84866f66453d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:17.987255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a615bb-d648-4050-81ef-84866f66453d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:18.008378 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ae14b-7a10-407a-9d04-b406147fd9ca None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:18.020311 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 825/1646] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:17:18.022457 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ae14b-7a10-407a-9d04-b406147fd9ca None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:18.027808 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960b95e-9203-4296-9202-1528012bd8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:18.028890 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960b95e-9203-4296-9202-1528012bd8ad admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:18.029374 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960b95e-9203-4296-9202-1528012bd8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:18.029880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5960b95e-9203-4296-9202-1528012bd8ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:18.030944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5960b95e-9203-4296-9202-1528012bd8ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:18.033223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5960b95e-9203-4296-9202-1528012bd8ad admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:18.085813 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a615bb-d648-4050-81ef-84866f66453d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:18.089172 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-82a615bb-d648-4050-81ef-84866f66453d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:18.107537 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a615bb-d648-4050-81ef-84866f66453d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:18.114013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d342b-921a-4644-bbf9-9e1cd853c5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:18.114938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d342b-921a-4644-bbf9-9e1cd853c5e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:18.117775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d342b-921a-4644-bbf9-9e1cd853c5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:18.117775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d342b-921a-4644-bbf9-9e1cd853c5e6 admin admin] PATH_INFO: `/v3/projects/570a83d81ade40c08d677db7c3ad102f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:18.117775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d342b-921a-4644-bbf9-9e1cd853c5e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=570a83d81ade40c08d677db7c3ad102f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:18.120365 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d342b-921a-4644-bbf9-9e1cd853c5e6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:18.149416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 822/1647] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:17:18 2022] POST /identity/v3/projects => generated 403 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:18.153845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960b95e-9203-4296-9202-1528012bd8ad admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:18.192286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:18.307612 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f00d342b-921a-4644-bbf9-9e1cd853c5e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '570a83d81ade40c08d677db7c3ad102f', 'request_id': 'req-f00d342b-921a-4644-bbf9-9e1cd853c5e6'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:18.319438 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5960b95e-9203-4296-9202-1528012bd8ad admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:18.326613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f00d342b-921a-4644-bbf9-9e1cd853c5e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '570a83d81ade40c08d677db7c3ad102f', 'request_id': 'req-f00d342b-921a-4644-bbf9-9e1cd853c5e6'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:18.341263 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f00d342b-921a-4644-bbf9-9e1cd853c5e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '570a83d81ade40c08d677db7c3ad102f', 'request_id': 'req-f00d342b-921a-4644-bbf9-9e1cd853c5e6'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:18.427684 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af343c1-1530-4022-a7f6-49b3f14947a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:18.428659 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af343c1-1530-4022-a7f6-49b3f14947a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:18.429159 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af343c1-1530-4022-a7f6-49b3f14947a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:18.429631 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af343c1-1530-4022-a7f6-49b3f14947a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:18.430783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af343c1-1530-4022-a7f6-49b3f14947a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:18.433209 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af343c1-1530-4022-a7f6-49b3f14947a2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:18.443206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f00d342b-921a-4644-bbf9-9e1cd853c5e6 admin admin] The token cache is being invalidate because project 570a83d81ade40c08d677db7c3ad102f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:17:18.443744 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f00d342b-921a-4644-bbf9-9e1cd853c5e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:18.444885 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f00d342b-921a-4644-bbf9-9e1cd853c5e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:18.448676 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 826/1648] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:17:18 2022] DELETE /identity/v3/projects/570a83d81ade40c08d677db7c3ad102f => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:18.452653 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d342b-921a-4644-bbf9-9e1cd853c5e6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:18.454610 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f00d342b-921a-4644-bbf9-9e1cd853c5e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:18.473529 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d342b-921a-4644-bbf9-9e1cd853c5e6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:18.482742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a69e28-3590-4b91-a14f-e3d89a5a6985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:18.484138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a69e28-3590-4b91-a14f-e3d89a5a6985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:18.484875 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a69e28-3590-4b91-a14f-e3d89a5a6985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:18.491762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a69e28-3590-4b91-a14f-e3d89a5a6985 admin admin] PATH_INFO: `/v3/users/255d98b204724decac5de6d284b9f2ce` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:18.497483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a69e28-3590-4b91-a14f-e3d89a5a6985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=255d98b204724decac5de6d284b9f2ce)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:18.497483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a69e28-3590-4b91-a14f-e3d89a5a6985 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:18.536775 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 823/1649] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:17:18 2022] POST /identity/v3/users => generated 302 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:18.551518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af343c1-1530-4022-a7f6-49b3f14947a2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:18.559751 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6af343c1-1530-4022-a7f6-49b3f14947a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:18.615931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af343c1-1530-4022-a7f6-49b3f14947a2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:18.633247 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015ade3-3fb3-4a54-9408-703041f34b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:18.634016 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015ade3-3fb3-4a54-9408-703041f34b1e admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:18.636564 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015ade3-3fb3-4a54-9408-703041f34b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:18.636918 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1015ade3-3fb3-4a54-9408-703041f34b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:18.638340 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015ade3-3fb3-4a54-9408-703041f34b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:18.642436 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1015ade3-3fb3-4a54-9408-703041f34b1e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:18.676860 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 824/1650] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:18.678731 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015ade3-3fb3-4a54-9408-703041f34b1e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:18.704756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1015ade3-3fb3-4a54-9408-703041f34b1e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:18.717515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61eceb-a7ea-4138-aa0d-ed0e195fe66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:18.718982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61eceb-a7ea-4138-aa0d-ed0e195fe66d admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:18.719269 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61eceb-a7ea-4138-aa0d-ed0e195fe66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:18.719608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61eceb-a7ea-4138-aa0d-ed0e195fe66d admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:18.721078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61eceb-a7ea-4138-aa0d-ed0e195fe66d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:18.725740 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61eceb-a7ea-4138-aa0d-ed0e195fe66d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:18.767206 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 825/1651] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:18.775443 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61eceb-a7ea-4138-aa0d-ed0e195fe66d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:18.842069 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61eceb-a7ea-4138-aa0d-ed0e195fe66d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:18.859777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416890fa-b776-46b4-a947-dd12b8b211bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:18.861654 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416890fa-b776-46b4-a947-dd12b8b211bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:18.862101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416890fa-b776-46b4-a947-dd12b8b211bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:18.862648 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416890fa-b776-46b4-a947-dd12b8b211bb admin admin] PATH_INFO: `/v3/projects/b290faf13175426ebf96674090f5e235/users/8387a23d47024d898746d88a0a787c8c/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:18.907922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-28a69e28-3590-4b91-a14f-e3d89a5a6985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '255d98b204724decac5de6d284b9f2ce'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:18.915504 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416890fa-b776-46b4-a947-dd12b8b211bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b290faf13175426ebf96674090f5e235, user_id=8387a23d47024d898746d88a0a787c8c, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:18.931393 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416890fa-b776-46b4-a947-dd12b8b211bb admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:18.971084 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-28a69e28-3590-4b91-a14f-e3d89a5a6985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '255d98b204724decac5de6d284b9f2ce'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:18.972022 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-28a69e28-3590-4b91-a14f-e3d89a5a6985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '255d98b204724decac5de6d284b9f2ce'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:18.972739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-28a69e28-3590-4b91-a14f-e3d89a5a6985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '255d98b204724decac5de6d284b9f2ce'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:18.996180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-28a69e28-3590-4b91-a14f-e3d89a5a6985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '255d98b204724decac5de6d284b9f2ce'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:19.073563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 826/1652] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:18 2022] PUT /identity/v3/projects/b290faf13175426ebf96674090f5e235/users/8387a23d47024d898746d88a0a787c8c/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:17:19.097779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5fb69-6da5-4741-9d67-30af2571151e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:19.098080 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5fb69-6da5-4741-9d67-30af2571151e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:19.098714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5fb69-6da5-4741-9d67-30af2571151e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:19.165906 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d1f5fb69-6da5-4741-9d67-30af2571151e None None] MFA Rules not processed for user `8387a23d47024d898746d88a0a787c8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:19.172312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d1f5fb69-6da5-4741-9d67-30af2571151e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:19.358121 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 827/1653] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:19 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:19.360973 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f5fb69-6da5-4741-9d67-30af2571151e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:19.365440 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d1f5fb69-6da5-4741-9d67-30af2571151e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:19.488845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f5fb69-6da5-4741-9d67-30af2571151e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:19.494982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8fa07c-67ca-44ba-8474-670137aaf62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:19.497498 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fa07c-67ca-44ba-8474-670137aaf62b admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:19.497498 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fa07c-67ca-44ba-8474-670137aaf62b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:19.497660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8fa07c-67ca-44ba-8474-670137aaf62b admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:19.499483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8fa07c-67ca-44ba-8474-670137aaf62b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:19.501555 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8fa07c-67ca-44ba-8474-670137aaf62b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:19.863378 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 827/1654] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:17:18 2022] DELETE /identity/v3/users/255d98b204724decac5de6d284b9f2ce => generated 0 bytes in 1404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:17:19.986891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 828/1655] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:17:19 2022] POST /identity/v3/projects => generated 403 bytes in 627 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:17:19.991301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a69e28-3590-4b91-a14f-e3d89a5a6985 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:20.049195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:20.135685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a69e28-3590-4b91-a14f-e3d89a5a6985 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:20.241785 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85a56e-863d-449b-83db-a846f478866a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:20.243297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85a56e-863d-449b-83db-a846f478866a admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:20.243732 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85a56e-863d-449b-83db-a846f478866a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:20.244153 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85a56e-863d-449b-83db-a846f478866a admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:20.245025 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85a56e-863d-449b-83db-a846f478866a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:20.247293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85a56e-863d-449b-83db-a846f478866a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:20.368736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 828/1656] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:17:19 2022] POST /identity/v3/users => generated 308 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:20.372416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8fa07c-67ca-44ba-8474-670137aaf62b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:20.385302 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8fa07c-67ca-44ba-8474-670137aaf62b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:20.419375 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9332f-a233-4e9b-9c9d-474dadc955dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:20.424280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9332f-a233-4e9b-9c9d-474dadc955dd admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:20.432342 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9332f-a233-4e9b-9c9d-474dadc955dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:20.432342 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9332f-a233-4e9b-9c9d-474dadc955dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:20.433334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d9332f-a233-4e9b-9c9d-474dadc955dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:20.435973 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d9332f-a233-4e9b-9c9d-474dadc955dd admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:20.470028 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 829/1657] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:20.471614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85a56e-863d-449b-83db-a846f478866a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:20.501255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85a56e-863d-449b-83db-a846f478866a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:20.513087 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f0e69-6853-4a98-95cb-f278d14cbf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:20.514218 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f0e69-6853-4a98-95cb-f278d14cbf37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:20.515202 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f0e69-6853-4a98-95cb-f278d14cbf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:20.517119 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f0e69-6853-4a98-95cb-f278d14cbf37 admin admin] PATH_INFO: `/v3/projects/1edd8299e0e947d393e5d337e85f081a/users/5fa865ec78d04870a2bce1c7ea99e0c8/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:20.581092 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f0e69-6853-4a98-95cb-f278d14cbf37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1edd8299e0e947d393e5d337e85f081a, user_id=5fa865ec78d04870a2bce1c7ea99e0c8, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:20.585414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8f0e69-6853-4a98-95cb-f278d14cbf37 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:20.686953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 829/1658] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:20 2022] PUT /identity/v3/projects/1edd8299e0e947d393e5d337e85f081a/users/5fa865ec78d04870a2bce1c7ea99e0c8/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:20.692609 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9332f-a233-4e9b-9c9d-474dadc955dd admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:20.710919 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9332f-a233-4e9b-9c9d-474dadc955dd admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:20.880967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477de5f-1cc3-4bf1-a27c-34a906d206f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:20.880967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477de5f-1cc3-4bf1-a27c-34a906d206f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:20.880967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477de5f-1cc3-4bf1-a27c-34a906d206f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:20.880967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477de5f-1cc3-4bf1-a27c-34a906d206f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:20.881281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477de5f-1cc3-4bf1-a27c-34a906d206f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:20.882851 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8477de5f-1cc3-4bf1-a27c-34a906d206f0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:20.922427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 830/1659] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:20.926529 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f0e69-6853-4a98-95cb-f278d14cbf37 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:20.939240 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8f0e69-6853-4a98-95cb-f278d14cbf37 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:20.946734 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b42c0-e224-4536-be5a-87b6ae6cfe80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:20.947660 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b42c0-e224-4536-be5a-87b6ae6cfe80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:20.953627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477de5f-1cc3-4bf1-a27c-34a906d206f0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:20.954053 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b42c0-e224-4536-be5a-87b6ae6cfe80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:20.954392 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b42c0-e224-4536-be5a-87b6ae6cfe80 admin admin] PATH_INFO: `/v3/projects/1edd8299e0e947d393e5d337e85f081a/users/5fa865ec78d04870a2bce1c7ea99e0c8/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:20.958782 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8477de5f-1cc3-4bf1-a27c-34a906d206f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:20.963693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b42c0-e224-4536-be5a-87b6ae6cfe80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1edd8299e0e947d393e5d337e85f081a, user_id=5fa865ec78d04870a2bce1c7ea99e0c8, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:20.967248 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b42c0-e224-4536-be5a-87b6ae6cfe80 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:20.989984 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 830/1660] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:20 2022] PUT /identity/v3/projects/1edd8299e0e947d393e5d337e85f081a/users/5fa865ec78d04870a2bce1c7ea99e0c8/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:20.995464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554dc7b-e54c-4c18-a292-ed1baa24c70e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:20.995464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554dc7b-e54c-4c18-a292-ed1baa24c70e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:20.995464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554dc7b-e54c-4c18-a292-ed1baa24c70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:20.995787 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8477de5f-1cc3-4bf1-a27c-34a906d206f0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:21.042505 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b554dc7b-e54c-4c18-a292-ed1baa24c70e None None] MFA Rules not processed for user `5fa865ec78d04870a2bce1c7ea99e0c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:21.044728 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b554dc7b-e54c-4c18-a292-ed1baa24c70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:21.059316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:21.103758 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785c397-8061-4a6b-b2b9-9369423250e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:21.104845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785c397-8061-4a6b-b2b9-9369423250e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:21.105228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785c397-8061-4a6b-b2b9-9369423250e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:21.105604 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785c397-8061-4a6b-b2b9-9369423250e0 admin admin] PATH_INFO: `/v3/projects/2e1e3c6f45844e038e95f047768a0fff` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:21.107017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785c397-8061-4a6b-b2b9-9369423250e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e1e3c6f45844e038e95f047768a0fff)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:21.110567 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785c397-8061-4a6b-b2b9-9369423250e0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:21.175571 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 831/1661] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:20 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:21.190590 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554dc7b-e54c-4c18-a292-ed1baa24c70e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:21.192278 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b554dc7b-e54c-4c18-a292-ed1baa24c70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:21.276765 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e785c397-8061-4a6b-b2b9-9369423250e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e1e3c6f45844e038e95f047768a0fff', 'request_id': 'req-e785c397-8061-4a6b-b2b9-9369423250e0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:21.315778 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554dc7b-e54c-4c18-a292-ed1baa24c70e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:21.345417 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e785c397-8061-4a6b-b2b9-9369423250e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e1e3c6f45844e038e95f047768a0fff', 'request_id': 'req-e785c397-8061-4a6b-b2b9-9369423250e0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:21.354343 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e785c397-8061-4a6b-b2b9-9369423250e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e1e3c6f45844e038e95f047768a0fff', 'request_id': 'req-e785c397-8061-4a6b-b2b9-9369423250e0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:21.379060 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e785c397-8061-4a6b-b2b9-9369423250e0 admin admin] The token cache is being invalidate because project 2e1e3c6f45844e038e95f047768a0fff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:17:21.387547 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e785c397-8061-4a6b-b2b9-9369423250e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:21.387547 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e785c397-8061-4a6b-b2b9-9369423250e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:21.387547 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 831/1662] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:17:20 2022] DELETE /identity/v3/projects/2e1e3c6f45844e038e95f047768a0fff => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:17:21.392531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009d2f5-eb45-4e9d-8a05-6fad19d9f1b1 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:21.393080 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009d2f5-eb45-4e9d-8a05-6fad19d9f1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:21.393714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8009d2f5-eb45-4e9d-8a05-6fad19d9f1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:21.396500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570cc7a4-cce3-4dae-a6c5-8d27544a405e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:21.397172 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570cc7a4-cce3-4dae-a6c5-8d27544a405e service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:21.397462 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570cc7a4-cce3-4dae-a6c5-8d27544a405e service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:21.397739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570cc7a4-cce3-4dae-a6c5-8d27544a405e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:21.406930 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-570cc7a4-cce3-4dae-a6c5-8d27544a405e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:21.451422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8009d2f5-eb45-4e9d-8a05-6fad19d9f1b1 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:21.451422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8009d2f5-eb45-4e9d-8a05-6fad19d9f1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:21.518630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570cc7a4-cce3-4dae-a6c5-8d27544a405e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:21.523081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570cc7a4-cce3-4dae-a6c5-8d27544a405e service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:21.562568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 832/1663] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:21 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:21.674133 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 832/1664] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:21.677067 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1aee2-87de-43c9-9d43-a507a60519ce None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:21.687841 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1aee2-87de-43c9-9d43-a507a60519ce None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:21.687841 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1aee2-87de-43c9-9d43-a507a60519ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:21.782709 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-dcb1aee2-87de-43c9-9d43-a507a60519ce None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:21.782709 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dcb1aee2-87de-43c9-9d43-a507a60519ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:21.797843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 833/1665] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:21.800788 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009d2f5-eb45-4e9d-8a05-6fad19d9f1b1 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:21.816095 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8009d2f5-eb45-4e9d-8a05-6fad19d9f1b1 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:21.851778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db62fd-4be3-4a3b-81b6-6581ccd79fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:21.851873 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db62fd-4be3-4a3b-81b6-6581ccd79fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:21.852474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db62fd-4be3-4a3b-81b6-6581ccd79fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:21.852474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db62fd-4be3-4a3b-81b6-6581ccd79fb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:21.853193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db62fd-4be3-4a3b-81b6-6581ccd79fb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:17:21.853518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db62fd-4be3-4a3b-81b6-6581ccd79fb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:21.855069 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db62fd-4be3-4a3b-81b6-6581ccd79fb8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:21.912614 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 833/1666] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:17:21.924102 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1aee2-87de-43c9-9d43-a507a60519ce None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:21.935281 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1aee2-87de-43c9-9d43-a507a60519ce None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:21.946433 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6e502-6374-469a-a0b0-767209cc2cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:21.957773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6e502-6374-469a-a0b0-767209cc2cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:21.958268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6e502-6374-469a-a0b0-767209cc2cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:21.958718 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6e502-6374-469a-a0b0-767209cc2cf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:21.960155 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6e502-6374-469a-a0b0-767209cc2cf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:21.962828 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6e502-6374-469a-a0b0-767209cc2cf5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:22.025502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:22.106445 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 834/1667] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:17:21 2022] POST /identity/v3/projects => generated 357 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:22.111018 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db62fd-4be3-4a3b-81b6-6581ccd79fb8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:22.132749 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db62fd-4be3-4a3b-81b6-6581ccd79fb8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:22.195242 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6e502-6374-469a-a0b0-767209cc2cf5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:22.197004 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-40c6e502-6374-469a-a0b0-767209cc2cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:22.292479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:22.307113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68d507-bfe7-4fcb-a652-e6b409b8f249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:22.308144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68d507-bfe7-4fcb-a652-e6b409b8f249 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:22.309413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68d507-bfe7-4fcb-a652-e6b409b8f249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:22.311159 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af68d507-bfe7-4fcb-a652-e6b409b8f249 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:22.313422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68d507-bfe7-4fcb-a652-e6b409b8f249 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:22.318091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af68d507-bfe7-4fcb-a652-e6b409b8f249 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:22.370654 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6e502-6374-469a-a0b0-767209cc2cf5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:22.378399 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ec7bc-9255-4b86-adc4-7cc6beeb5b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:22.379327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ec7bc-9255-4b86-adc4-7cc6beeb5b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:22.379618 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ec7bc-9255-4b86-adc4-7cc6beeb5b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:22.379901 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ec7bc-9255-4b86-adc4-7cc6beeb5b9a admin admin] PATH_INFO: `/v3/users/8387a23d47024d898746d88a0a787c8c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:22.381245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ec7bc-9255-4b86-adc4-7cc6beeb5b9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8387a23d47024d898746d88a0a787c8c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:22.384906 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ec7bc-9255-4b86-adc4-7cc6beeb5b9a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:22.461709 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 834/1668] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:17:22 2022] POST /identity/v3/users => generated 279 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:22.465970 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68d507-bfe7-4fcb-a652-e6b409b8f249 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:22.486996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af68d507-bfe7-4fcb-a652-e6b409b8f249 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:22.502343 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ab7e6-4917-4020-a60a-eafa16120ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:22.503162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ab7e6-4917-4020-a60a-eafa16120ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:22.503455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ab7e6-4917-4020-a60a-eafa16120ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:22.503736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ab7e6-4917-4020-a60a-eafa16120ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:22.504535 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ab7e6-4917-4020-a60a-eafa16120ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:22.510632 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ab7e6-4917-4020-a60a-eafa16120ad5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:22.535995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 835/1669] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:22.539850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ab7e6-4917-4020-a60a-eafa16120ad5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:22.555798 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ab7e6-4917-4020-a60a-eafa16120ad5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:22.580545 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203e88a-3a79-4d16-8154-76e73449b8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:22.581298 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203e88a-3a79-4d16-8154-76e73449b8bd admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:22.582096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203e88a-3a79-4d16-8154-76e73449b8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:22.582096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203e88a-3a79-4d16-8154-76e73449b8bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:22.582682 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203e88a-3a79-4d16-8154-76e73449b8bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:22.584694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203e88a-3a79-4d16-8154-76e73449b8bd admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:22.607336 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 836/1670] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:22.611969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203e88a-3a79-4d16-8154-76e73449b8bd admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:22.635905 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203e88a-3a79-4d16-8154-76e73449b8bd admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:22.655135 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ec82a-3e57-4fd9-81b0-42454e0ffd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:22.656344 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ec82a-3e57-4fd9-81b0-42454e0ffd4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:22.656740 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ec82a-3e57-4fd9-81b0-42454e0ffd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:22.657161 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ec82a-3e57-4fd9-81b0-42454e0ffd4d admin admin] PATH_INFO: `/v3/projects/b7cce4ceff8f42b9bf990acd1127509d/users/942e1e702362438384e98e0474353023/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:22.802312 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c80ec7bc-9255-4b86-adc4-7cc6beeb5b9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8387a23d47024d898746d88a0a787c8c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:22.836420 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c80ec7bc-9255-4b86-adc4-7cc6beeb5b9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8387a23d47024d898746d88a0a787c8c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:22.839545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c80ec7bc-9255-4b86-adc4-7cc6beeb5b9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8387a23d47024d898746d88a0a787c8c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:22.841583 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c80ec7bc-9255-4b86-adc4-7cc6beeb5b9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8387a23d47024d898746d88a0a787c8c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:22.869554 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c80ec7bc-9255-4b86-adc4-7cc6beeb5b9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8387a23d47024d898746d88a0a787c8c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:22.959349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 835/1671] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:17:22 2022] DELETE /identity/v3/users/8387a23d47024d898746d88a0a787c8c => generated 0 bytes in 760 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:22.961753 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd8aaf-876b-4b52-93c0-d1c4261a53bd None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:22.963716 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd8aaf-876b-4b52-93c0-d1c4261a53bd None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:22.963855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ec82a-3e57-4fd9-81b0-42454e0ffd4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7cce4ceff8f42b9bf990acd1127509d, user_id=942e1e702362438384e98e0474353023, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:22.964209 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddd8aaf-876b-4b52-93c0-d1c4261a53bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:22.966731 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ec82a-3e57-4fd9-81b0-42454e0ffd4d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:23.024673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6ddd8aaf-876b-4b52-93c0-d1c4261a53bd None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:23.027110 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6ddd8aaf-876b-4b52-93c0-d1c4261a53bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:23.059150 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 837/1672] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:22 2022] PUT /identity/v3/projects/b7cce4ceff8f42b9bf990acd1127509d/users/942e1e702362438384e98e0474353023/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:23.080047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510121f0-473e-4f6d-9a34-6ea418ed69f2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:23.080387 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510121f0-473e-4f6d-9a34-6ea418ed69f2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:23.080673 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510121f0-473e-4f6d-9a34-6ea418ed69f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:23.136436 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-510121f0-473e-4f6d-9a34-6ea418ed69f2 None None] MFA Rules not processed for user `942e1e702362438384e98e0474353023`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:23.137359 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-510121f0-473e-4f6d-9a34-6ea418ed69f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:23.202283 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 836/1673] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:23.261041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd8aaf-876b-4b52-93c0-d1c4261a53bd None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:23.263463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6ddd8aaf-876b-4b52-93c0-d1c4261a53bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:23.292663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 838/1674] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:23 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:23.293519 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510121f0-473e-4f6d-9a34-6ea418ed69f2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:23.295879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-510121f0-473e-4f6d-9a34-6ea418ed69f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:23.357701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:23.424180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510121f0-473e-4f6d-9a34-6ea418ed69f2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:23.429586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ba84c-db2e-4fd7-9280-d59cc7b80db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:23.430270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ba84c-db2e-4fd7-9280-d59cc7b80db2 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:23.430575 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ba84c-db2e-4fd7-9280-d59cc7b80db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:23.430855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ba84c-db2e-4fd7-9280-d59cc7b80db2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:23.431560 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ba84c-db2e-4fd7-9280-d59cc7b80db2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:23.433541 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ba84c-db2e-4fd7-9280-d59cc7b80db2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:23.463321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddd8aaf-876b-4b52-93c0-d1c4261a53bd None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:23.566625 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 839/1675] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:17:23 2022] POST /identity/v3/projects => generated 355 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:23.567546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ba84c-db2e-4fd7-9280-d59cc7b80db2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:23.616366 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:23.627181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72532351-39b2-4f5a-b60f-5c7cd951223e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:23.630890 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72532351-39b2-4f5a-b60f-5c7cd951223e service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:23.631388 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72532351-39b2-4f5a-b60f-5c7cd951223e service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:23.631910 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72532351-39b2-4f5a-b60f-5c7cd951223e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:23.682419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72532351-39b2-4f5a-b60f-5c7cd951223e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:23.683912 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72532351-39b2-4f5a-b60f-5c7cd951223e service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:23.724294 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 837/1676] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:23.735377 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ba84c-db2e-4fd7-9280-d59cc7b80db2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:23.749806 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7571ef1-cf59-4cfc-af46-d0f46dad022d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:23.750538 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7571ef1-cf59-4cfc-af46-d0f46dad022d admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:23.750826 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7571ef1-cf59-4cfc-af46-d0f46dad022d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:23.751103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7571ef1-cf59-4cfc-af46-d0f46dad022d admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:23.751964 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7571ef1-cf59-4cfc-af46-d0f46dad022d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:23.753867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7571ef1-cf59-4cfc-af46-d0f46dad022d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:23.914510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72532351-39b2-4f5a-b60f-5c7cd951223e service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:23.915215 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 840/1677] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:17:23 2022] POST /identity/v3/users => generated 284 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:23.925233 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72532351-39b2-4f5a-b60f-5c7cd951223e service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:23.944113 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd08f0-e83b-45f5-8db8-12d2a21d883c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:23.944805 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd08f0-e83b-45f5-8db8-12d2a21d883c admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:23.945095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd08f0-e83b-45f5-8db8-12d2a21d883c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:23.945374 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd08f0-e83b-45f5-8db8-12d2a21d883c admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:23.946120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd08f0-e83b-45f5-8db8-12d2a21d883c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:23.951444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd08f0-e83b-45f5-8db8-12d2a21d883c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:23.977818 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 838/1678] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:23.982131 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7571ef1-cf59-4cfc-af46-d0f46dad022d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:23.998281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7571ef1-cf59-4cfc-af46-d0f46dad022d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:24.016577 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdec3d-665a-44af-b19a-cf70949b1dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:24.017314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdec3d-665a-44af-b19a-cf70949b1dd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:24.017725 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdec3d-665a-44af-b19a-cf70949b1dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:24.018016 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdec3d-665a-44af-b19a-cf70949b1dd9 admin admin] PATH_INFO: `/v3/projects/448d79b9f1884604ac5d18edda555fd0/users/0f7395f7dbdd4faf92dd98f963bedeb2/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:24.045333 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdec3d-665a-44af-b19a-cf70949b1dd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=448d79b9f1884604ac5d18edda555fd0, user_id=0f7395f7dbdd4faf92dd98f963bedeb2, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:24.049198 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdec3d-665a-44af-b19a-cf70949b1dd9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:24.138728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 841/1679] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:23 2022] PUT /identity/v3/projects/448d79b9f1884604ac5d18edda555fd0/users/0f7395f7dbdd4faf92dd98f963bedeb2/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:17:24.145899 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd08f0-e83b-45f5-8db8-12d2a21d883c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:24.158413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd08f0-e83b-45f5-8db8-12d2a21d883c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:24.240245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7425ef58-0997-433b-b792-3c4ce9a11553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:24.241569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425ef58-0997-433b-b792-3c4ce9a11553 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:24.242121 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425ef58-0997-433b-b792-3c4ce9a11553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:24.242651 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425ef58-0997-433b-b792-3c4ce9a11553 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:24.243680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7425ef58-0997-433b-b792-3c4ce9a11553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:24.245661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7425ef58-0997-433b-b792-3c4ce9a11553 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:24.263808 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdec3d-665a-44af-b19a-cf70949b1dd9 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:24.272628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 839/1680] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:24 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:24.281645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdec3d-665a-44af-b19a-cf70949b1dd9 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:24.290204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf62a7a-44ca-4fc2-83c2-1869443980a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:24.291332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf62a7a-44ca-4fc2-83c2-1869443980a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:24.291621 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf62a7a-44ca-4fc2-83c2-1869443980a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:24.291956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf62a7a-44ca-4fc2-83c2-1869443980a0 admin admin] PATH_INFO: `/v3/projects/448d79b9f1884604ac5d18edda555fd0/users/0f7395f7dbdd4faf92dd98f963bedeb2/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:24.295061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf62a7a-44ca-4fc2-83c2-1869443980a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=448d79b9f1884604ac5d18edda555fd0, user_id=0f7395f7dbdd4faf92dd98f963bedeb2, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:24.300615 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf62a7a-44ca-4fc2-83c2-1869443980a0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:24.331775 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 842/1681] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:24 2022] PUT /identity/v3/projects/448d79b9f1884604ac5d18edda555fd0/users/0f7395f7dbdd4faf92dd98f963bedeb2/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:24.344995 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e376bd-0080-4a83-8b98-75f4aed2ac1b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:24.345302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e376bd-0080-4a83-8b98-75f4aed2ac1b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:24.345586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e376bd-0080-4a83-8b98-75f4aed2ac1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:24.408292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-64e376bd-0080-4a83-8b98-75f4aed2ac1b None None] MFA Rules not processed for user `0f7395f7dbdd4faf92dd98f963bedeb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:24.409542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-64e376bd-0080-4a83-8b98-75f4aed2ac1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:24.579973 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 840/1682] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:24 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:24.588569 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf62a7a-44ca-4fc2-83c2-1869443980a0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:24.590942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bdf62a7a-44ca-4fc2-83c2-1869443980a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:24.713702 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:24.740917 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf62a7a-44ca-4fc2-83c2-1869443980a0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:24.751443 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be705483-6e26-4781-9f43-1328f7b1e077 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:24.752185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be705483-6e26-4781-9f43-1328f7b1e077 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:24.752468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be705483-6e26-4781-9f43-1328f7b1e077 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:24.752741 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be705483-6e26-4781-9f43-1328f7b1e077 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:24.775312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be705483-6e26-4781-9f43-1328f7b1e077 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:24.777458 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be705483-6e26-4781-9f43-1328f7b1e077 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:24.826692 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 843/1683] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:24 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:24.840758 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:25.166006 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e376bd-0080-4a83-8b98-75f4aed2ac1b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:25.184377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e376bd-0080-4a83-8b98-75f4aed2ac1b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:25.193129 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a2a62-1d9a-4523-b12e-2f6b7f2c39d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:25.193808 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a2a62-1d9a-4523-b12e-2f6b7f2c39d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:25.194093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a2a62-1d9a-4523-b12e-2f6b7f2c39d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:25.194391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601a2a62-1d9a-4523-b12e-2f6b7f2c39d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:25.317811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a2a62-1d9a-4523-b12e-2f6b7f2c39d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:25.319339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601a2a62-1d9a-4523-b12e-2f6b7f2c39d7 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:25.340418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 841/1684] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:25 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:26.409101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be705483-6e26-4781-9f43-1328f7b1e077 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:26.409101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-be705483-6e26-4781-9f43-1328f7b1e077 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:26.552196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be705483-6e26-4781-9f43-1328f7b1e077 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:26.563850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76985a-32e0-484e-b754-9068ba5ac8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:26.564949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76985a-32e0-484e-b754-9068ba5ac8e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:26.565950 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76985a-32e0-484e-b754-9068ba5ac8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:26.566297 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76985a-32e0-484e-b754-9068ba5ac8e3 admin admin] PATH_INFO: `/v3/projects/b290faf13175426ebf96674090f5e235` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:26.567660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76985a-32e0-484e-b754-9068ba5ac8e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b290faf13175426ebf96674090f5e235)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:26.569632 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76985a-32e0-484e-b754-9068ba5ac8e3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:26.669287 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:26.728841 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:26.743959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8f76985a-32e0-484e-b754-9068ba5ac8e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b290faf13175426ebf96674090f5e235', 'request_id': 'req-8f76985a-32e0-484e-b754-9068ba5ac8e3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:26.830256 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8f76985a-32e0-484e-b754-9068ba5ac8e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b290faf13175426ebf96674090f5e235', 'request_id': 'req-8f76985a-32e0-484e-b754-9068ba5ac8e3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:26.839494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8f76985a-32e0-484e-b754-9068ba5ac8e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b290faf13175426ebf96674090f5e235', 'request_id': 'req-8f76985a-32e0-484e-b754-9068ba5ac8e3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:26.867577 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8f76985a-32e0-484e-b754-9068ba5ac8e3 admin admin] The token cache is being invalidate because project b290faf13175426ebf96674090f5e235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:17:26.867881 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8f76985a-32e0-484e-b754-9068ba5ac8e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:26.872697 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8f76985a-32e0-484e-b754-9068ba5ac8e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:26.875759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 844/1685] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:17:26 2022] DELETE /identity/v3/projects/b290faf13175426ebf96674090f5e235 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:26.879536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a2a62-1d9a-4523-b12e-2f6b7f2c39d7 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:26.907589 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601a2a62-1d9a-4523-b12e-2f6b7f2c39d7 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:27.148251 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36e95e-17fe-4ae1-a5ab-00b7cf14cf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:27.149153 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36e95e-17fe-4ae1-a5ab-00b7cf14cf02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:27.149445 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36e95e-17fe-4ae1-a5ab-00b7cf14cf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:27.150024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36e95e-17fe-4ae1-a5ab-00b7cf14cf02 admin admin] PATH_INFO: `/v3/users/5fa865ec78d04870a2bce1c7ea99e0c8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:27.152686 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a36e95e-17fe-4ae1-a5ab-00b7cf14cf02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fa865ec78d04870a2bce1c7ea99e0c8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:27.155781 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a36e95e-17fe-4ae1-a5ab-00b7cf14cf02 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:27.231459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:27.483491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4a36e95e-17fe-4ae1-a5ab-00b7cf14cf02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fa865ec78d04870a2bce1c7ea99e0c8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:27.539579 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4a36e95e-17fe-4ae1-a5ab-00b7cf14cf02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fa865ec78d04870a2bce1c7ea99e0c8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:27.540834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4a36e95e-17fe-4ae1-a5ab-00b7cf14cf02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fa865ec78d04870a2bce1c7ea99e0c8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:27.541707 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4a36e95e-17fe-4ae1-a5ab-00b7cf14cf02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fa865ec78d04870a2bce1c7ea99e0c8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:27.571747 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-4a36e95e-17fe-4ae1-a5ab-00b7cf14cf02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fa865ec78d04870a2bce1c7ea99e0c8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:27.667459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 842/1686] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:17:26 2022] DELETE /identity/v3/users/5fa865ec78d04870a2bce1c7ea99e0c8 => generated 0 bytes in 788 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:28.052282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce9b22-3c90-4572-af04-4fe003ee5c39 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:28.053281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce9b22-3c90-4572-af04-4fe003ee5c39 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:28.053720 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce9b22-3c90-4572-af04-4fe003ee5c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:28.138935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e0ce9b22-3c90-4572-af04-4fe003ee5c39 None None] MFA Rules not processed for user `aa8237a791034b80973d5cc4329d015c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:28.140711 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e0ce9b22-3c90-4572-af04-4fe003ee5c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:28.406201 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 845/1687] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:28.409129 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cca2b-ceae-4b12-8e59-78740822a7a9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:28.409695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cca2b-ceae-4b12-8e59-78740822a7a9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:28.410165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cca2b-ceae-4b12-8e59-78740822a7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:28.447706 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3a4cca2b-ceae-4b12-8e59-78740822a7a9 None None] MFA Rules not processed for user `aa8237a791034b80973d5cc4329d015c`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:28.448976 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3a4cca2b-ceae-4b12-8e59-78740822a7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:28.470185 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 843/1688] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:28.493026 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce9b22-3c90-4572-af04-4fe003ee5c39 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:28.494552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e0ce9b22-3c90-4572-af04-4fe003ee5c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:28.729262 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce9b22-3c90-4572-af04-4fe003ee5c39 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:28.745254 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191666d-729e-49d4-a682-91fcdd05325b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:28.745941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191666d-729e-49d4-a682-91fcdd05325b service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:28.746228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191666d-729e-49d4-a682-91fcdd05325b service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:28.746539 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191666d-729e-49d4-a682-91fcdd05325b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:28.771369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191666d-729e-49d4-a682-91fcdd05325b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:28.773647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191666d-729e-49d4-a682-91fcdd05325b service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:28.808498 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 846/1689] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:28.905950 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cca2b-ceae-4b12-8e59-78740822a7a9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:28.907455 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3a4cca2b-ceae-4b12-8e59-78740822a7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:29.211808 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cca2b-ceae-4b12-8e59-78740822a7a9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:29.218502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041fd440-cec3-4a6b-b1dd-14f190678f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:29.219652 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fd440-cec3-4a6b-b1dd-14f190678f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:29.220182 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fd440-cec3-4a6b-b1dd-14f190678f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:29.220629 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041fd440-cec3-4a6b-b1dd-14f190678f2d admin admin] PATH_INFO: `/v3/projects/1edd8299e0e947d393e5d337e85f081a` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:29.223211 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041fd440-cec3-4a6b-b1dd-14f190678f2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1edd8299e0e947d393e5d337e85f081a)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:29.228503 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041fd440-cec3-4a6b-b1dd-14f190678f2d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:29.435517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a020a7-5301-4307-9601-4d1443c6207a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:29.436033 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a020a7-5301-4307-9601-4d1443c6207a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:29.436140 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a020a7-5301-4307-9601-4d1443c6207a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:29.460586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-041fd440-cec3-4a6b-b1dd-14f190678f2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1edd8299e0e947d393e5d337e85f081a', 'request_id': 'req-041fd440-cec3-4a6b-b1dd-14f190678f2d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:29.509812 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-041fd440-cec3-4a6b-b1dd-14f190678f2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1edd8299e0e947d393e5d337e85f081a', 'request_id': 'req-041fd440-cec3-4a6b-b1dd-14f190678f2d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:29.528474 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-041fd440-cec3-4a6b-b1dd-14f190678f2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1edd8299e0e947d393e5d337e85f081a', 'request_id': 'req-041fd440-cec3-4a6b-b1dd-14f190678f2d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:29.536850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-32a020a7-5301-4307-9601-4d1443c6207a None None] MFA Rules not processed for user `9ffa029a3b1e4919b00e58a9737816d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:29.540849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-32a020a7-5301-4307-9601-4d1443c6207a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:29.660527 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:29.682766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-041fd440-cec3-4a6b-b1dd-14f190678f2d admin admin] The token cache is being invalidate because project 1edd8299e0e947d393e5d337e85f081a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:17:29.688967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-041fd440-cec3-4a6b-b1dd-14f190678f2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:29.688967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-041fd440-cec3-4a6b-b1dd-14f190678f2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:29.689538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 844/1690] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:17:28 2022] DELETE /identity/v3/projects/1edd8299e0e947d393e5d337e85f081a => generated 0 bytes in 784 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:29.699611 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025efa1-52c9-4413-87e6-74d14c25bf10 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:29.699901 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025efa1-52c9-4413-87e6-74d14c25bf10 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:29.702122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025efa1-52c9-4413-87e6-74d14c25bf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:29.764365 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4025efa1-52c9-4413-87e6-74d14c25bf10 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:29.772288 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4025efa1-52c9-4413-87e6-74d14c25bf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:29.806820 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 847/1691] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:29.813616 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:29.815537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a020a7-5301-4307-9601-4d1443c6207a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:29.817484 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-32a020a7-5301-4307-9601-4d1443c6207a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:29.950934 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 845/1692] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:29.965045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98fd57-8b31-4020-a825-f0da2c27a667 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:29.970805 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98fd57-8b31-4020-a825-f0da2c27a667 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:29.970805 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98fd57-8b31-4020-a825-f0da2c27a667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:30.012464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0c98fd57-8b31-4020-a825-f0da2c27a667 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:30.017206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0c98fd57-8b31-4020-a825-f0da2c27a667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:30.018680 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a020a7-5301-4307-9601-4d1443c6207a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:30.033468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce775a1-e675-4aa5-90f5-fb1c6c2f6e5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:30.034625 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce775a1-e675-4aa5-90f5-fb1c6c2f6e5b service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:30.040952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce775a1-e675-4aa5-90f5-fb1c6c2f6e5b service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:30.040952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce775a1-e675-4aa5-90f5-fb1c6c2f6e5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:30.050727 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 846/1693] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:30.063355 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98fd57-8b31-4020-a825-f0da2c27a667 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:30.084559 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce775a1-e675-4aa5-90f5-fb1c6c2f6e5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:30.087654 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce775a1-e675-4aa5-90f5-fb1c6c2f6e5b service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:30.094847 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98fd57-8b31-4020-a825-f0da2c27a667 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:30.111461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d32d8-a194-4155-b5fe-c3eed620378e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:30.112156 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d32d8-a194-4155-b5fe-c3eed620378e admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:30.112446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d32d8-a194-4155-b5fe-c3eed620378e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:30.112733 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d32d8-a194-4155-b5fe-c3eed620378e admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:30.113422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 848/1694] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:30.130065 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d32d8-a194-4155-b5fe-c3eed620378e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:17:30.130328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d32d8-a194-4155-b5fe-c3eed620378e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:30.134175 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d32d8-a194-4155-b5fe-c3eed620378e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:30.191283 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 847/1695] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:17:30.193020 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce775a1-e675-4aa5-90f5-fb1c6c2f6e5b service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:30.213889 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce775a1-e675-4aa5-90f5-fb1c6c2f6e5b service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:30.251099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f4dfb-5b08-460e-a4c6-5a48c084d605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:30.251858 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f4dfb-5b08-460e-a4c6-5a48c084d605 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:30.252326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f4dfb-5b08-460e-a4c6-5a48c084d605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:30.252567 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d32d8-a194-4155-b5fe-c3eed620378e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:30.252671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f4dfb-5b08-460e-a4c6-5a48c084d605 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:30.253298 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f4dfb-5b08-460e-a4c6-5a48c084d605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:30.254460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-630d32d8-a194-4155-b5fe-c3eed620378e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:30.256368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f4dfb-5b08-460e-a4c6-5a48c084d605 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:30.272939 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d32d8-a194-4155-b5fe-c3eed620378e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:30.280918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bbce5-51ba-4dbb-8300-2747310878d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:30.282054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bbce5-51ba-4dbb-8300-2747310878d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:30.282344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bbce5-51ba-4dbb-8300-2747310878d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:30.282758 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4bbce5-51ba-4dbb-8300-2747310878d2 admin admin] PATH_INFO: `/v3/users/50be5e96476a4a8fb6acb125d3f99841` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:30.284862 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bbce5-51ba-4dbb-8300-2747310878d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50be5e96476a4a8fb6acb125d3f99841)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:30.288480 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4bbce5-51ba-4dbb-8300-2747310878d2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:30.457404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 849/1696] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:17:30 2022] POST /identity/v3/projects => generated 405 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:30.460992 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34c978-0f9b-4ca7-8a7a-bc60b3e9a11f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:30.461286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34c978-0f9b-4ca7-8a7a-bc60b3e9a11f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:30.461570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34c978-0f9b-4ca7-8a7a-bc60b3e9a11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:30.551581 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-de34c978-0f9b-4ca7-8a7a-bc60b3e9a11f None None] MFA Rules not processed for user `114d227e9645409e9082cf9b09b05c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:30.552816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-de34c978-0f9b-4ca7-8a7a-bc60b3e9a11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:30.606115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9b4bbce5-51ba-4dbb-8300-2747310878d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50be5e96476a4a8fb6acb125d3f99841'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:30.661506 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9b4bbce5-51ba-4dbb-8300-2747310878d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50be5e96476a4a8fb6acb125d3f99841'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:30.664759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9b4bbce5-51ba-4dbb-8300-2747310878d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50be5e96476a4a8fb6acb125d3f99841'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:30.666797 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9b4bbce5-51ba-4dbb-8300-2747310878d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50be5e96476a4a8fb6acb125d3f99841'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:30.684037 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9b4bbce5-51ba-4dbb-8300-2747310878d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50be5e96476a4a8fb6acb125d3f99841'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:30.698976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 850/1697] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:30.713349 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34c978-0f9b-4ca7-8a7a-bc60b3e9a11f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:30.807087 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 848/1698] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:17:30 2022] DELETE /identity/v3/users/50be5e96476a4a8fb6acb125d3f99841 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:30.810531 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bbce5-51ba-4dbb-8300-2747310878d2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:30.814831 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9b4bbce5-51ba-4dbb-8300-2747310878d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:30.930704 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34c978-0f9b-4ca7-8a7a-bc60b3e9a11f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:31.100222 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4bbce5-51ba-4dbb-8300-2747310878d2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:31.116418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c9a2b-ed15-49fd-844b-28b5577475ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:31.117024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c9a2b-ed15-49fd-844b-28b5577475ea service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:31.117315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c9a2b-ed15-49fd-844b-28b5577475ea service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:31.126030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c9a2b-ed15-49fd-844b-28b5577475ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:31.132092 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b19c9a2b-ed15-49fd-844b-28b5577475ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:31.136635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff119282-5114-49cb-b60d-32275fb80e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:31.139420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff119282-5114-49cb-b60d-32275fb80e45 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:31.139728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff119282-5114-49cb-b60d-32275fb80e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:31.140056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff119282-5114-49cb-b60d-32275fb80e45 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:31.141040 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff119282-5114-49cb-b60d-32275fb80e45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:31.143796 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff119282-5114-49cb-b60d-32275fb80e45 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:31.242492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c9a2b-ed15-49fd-844b-28b5577475ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:31.244419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c9a2b-ed15-49fd-844b-28b5577475ea service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:31.259125 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 849/1699] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:31.263124 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe733b-ce51-444f-a744-e789cf393a74 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:31.263713 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe733b-ce51-444f-a744-e789cf393a74 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:31.269867 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe733b-ce51-444f-a744-e789cf393a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:31.289418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-06fe733b-ce51-444f-a744-e789cf393a74 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:31.291063 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-06fe733b-ce51-444f-a744-e789cf393a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:31.307753 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 851/1700] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:17:30 2022] POST /identity/v3/users => generated 303 bytes in 608 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:31.307753 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4dc8b-a792-45fc-96f7-601dd13830e5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:31.307753 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4dc8b-a792-45fc-96f7-601dd13830e5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:31.307753 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4dc8b-a792-45fc-96f7-601dd13830e5 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:31.315584 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 852/1701] 10.222.0.16 () {54 vars in 912 bytes} [Fri Jun 24 04:17:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:17:31.331486 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 850/1702] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:31.331749 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4dc8b-a792-45fc-96f7-601dd13830e5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:31.338463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9764a4-a3d0-4abf-8efa-b90919a06a60 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:31.338463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9764a4-a3d0-4abf-8efa-b90919a06a60 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:31.338463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9764a4-a3d0-4abf-8efa-b90919a06a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:31.338740 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-36a4dc8b-a792-45fc-96f7-601dd13830e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:31.371110 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4dc8b-a792-45fc-96f7-601dd13830e5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:31.379724 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fddb1-2693-4d80-88f3-93cf743b6fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:31.382762 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fddb1-2693-4d80-88f3-93cf743b6fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:31.383278 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fddb1-2693-4d80-88f3-93cf743b6fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:31.383739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fddb1-2693-4d80-88f3-93cf743b6fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:31.384698 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fddb1-2693-4d80-88f3-93cf743b6fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:31.386705 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fddb1-2693-4d80-88f3-93cf743b6fb0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:31.417779 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7c9764a4-a3d0-4abf-8efa-b90919a06a60 None None] MFA Rules not processed for user `fd98b54f587f4b42b4dd0a058b30c684`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:31.417779 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7c9764a4-a3d0-4abf-8efa-b90919a06a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:31.447442 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 853/1703] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:31.447442 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fddb1-2693-4d80-88f3-93cf743b6fb0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:31.447442 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d49fddb1-2693-4d80-88f3-93cf743b6fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:31.568669 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 851/1704] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jun 24 04:17:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:31.571857 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9764a4-a3d0-4abf-8efa-b90919a06a60 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:31.609013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9764a4-a3d0-4abf-8efa-b90919a06a60 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:31.634995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fddb1-2693-4d80-88f3-93cf743b6fb0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:31.639473 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f6391-2dc7-48ce-b6ad-5aec9d18690e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:31.641111 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f6391-2dc7-48ce-b6ad-5aec9d18690e admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:31.644850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f6391-2dc7-48ce-b6ad-5aec9d18690e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:31.648179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f6391-2dc7-48ce-b6ad-5aec9d18690e admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:31.649292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f69b3-f853-4c27-9262-a604172f82d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:31.649646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f6391-2dc7-48ce-b6ad-5aec9d18690e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:31.650046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f69b3-f853-4c27-9262-a604172f82d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:31.650332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f69b3-f853-4c27-9262-a604172f82d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:31.650631 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f69b3-f853-4c27-9262-a604172f82d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:31.653533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4f6391-2dc7-48ce-b6ad-5aec9d18690e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:31.682777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f69b3-f853-4c27-9262-a604172f82d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:31.684505 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f69b3-f853-4c27-9262-a604172f82d8 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:31.689226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 852/1705] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:31.694109 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f6391-2dc7-48ce-b6ad-5aec9d18690e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:31.722137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 854/1706] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:31.726822 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f6391-2dc7-48ce-b6ad-5aec9d18690e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:31.749317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7f70f-c5ae-4d15-84a4-42a1385967c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:31.749317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7f70f-c5ae-4d15-84a4-42a1385967c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:31.749317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7f70f-c5ae-4d15-84a4-42a1385967c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:31.749317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7f70f-c5ae-4d15-84a4-42a1385967c0 admin admin] PATH_INFO: `/v3/projects/f414fbc699c141fb9990984e06e90155/users/6d53a9bd8dd8494e98bf7bd730bac2e2/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:31.820683 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7f70f-c5ae-4d15-84a4-42a1385967c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f414fbc699c141fb9990984e06e90155, user_id=6d53a9bd8dd8494e98bf7bd730bac2e2, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:31.825559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7f70f-c5ae-4d15-84a4-42a1385967c0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:32.124266 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:32.127378 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 853/1707] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:31 2022] PUT /identity/v3/projects/f414fbc699c141fb9990984e06e90155/users/6d53a9bd8dd8494e98bf7bd730bac2e2/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:32.128305 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff325f55-d300-40e0-8a6f-c5475ebfc814 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:32.128305 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff325f55-d300-40e0-8a6f-c5475ebfc814 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:32.128305 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff325f55-d300-40e0-8a6f-c5475ebfc814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:32.242039 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ff325f55-d300-40e0-8a6f-c5475ebfc814 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:32.250190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ff325f55-d300-40e0-8a6f-c5475ebfc814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:32.373362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:32.595376 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 855/1708] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:32.661394 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7f70f-c5ae-4d15-84a4-42a1385967c0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:32.664182 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-15d7f70f-c5ae-4d15-84a4-42a1385967c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:32.667438 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff325f55-d300-40e0-8a6f-c5475ebfc814 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:32.669164 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ff325f55-d300-40e0-8a6f-c5475ebfc814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:32.946648 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff325f55-d300-40e0-8a6f-c5475ebfc814 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:32.953327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7f70f-c5ae-4d15-84a4-42a1385967c0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:32.962383 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80694fb7-1996-4f21-bb5c-e175e5aa2769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:32.963102 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8b2db-e47c-40ce-b659-db450771c9e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:32.963794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8b2db-e47c-40ce-b659-db450771c9e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:32.964088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8b2db-e47c-40ce-b659-db450771c9e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:32.965237 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80694fb7-1996-4f21-bb5c-e175e5aa2769 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:32.965700 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80694fb7-1996-4f21-bb5c-e175e5aa2769 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:32.965840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80694fb7-1996-4f21-bb5c-e175e5aa2769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:32.967951 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8b2db-e47c-40ce-b659-db450771c9e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:33.006994 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8b2db-e47c-40ce-b659-db450771c9e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:33.006994 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8b2db-e47c-40ce-b659-db450771c9e5 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:33.009074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80694fb7-1996-4f21-bb5c-e175e5aa2769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:33.010861 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80694fb7-1996-4f21-bb5c-e175e5aa2769 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:33.027795 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 854/1709] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:33.045168 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 856/1710] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:33.183216 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8b2db-e47c-40ce-b659-db450771c9e5 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:33.184712 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b3d8b2db-e47c-40ce-b659-db450771c9e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:33.214914 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8b2db-e47c-40ce-b659-db450771c9e5 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:33.239385 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fde45-b5ae-42ca-8d3f-8ee90ec16d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:33.239385 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fde45-b5ae-42ca-8d3f-8ee90ec16d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:33.239385 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fde45-b5ae-42ca-8d3f-8ee90ec16d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:33.239385 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fde45-b5ae-42ca-8d3f-8ee90ec16d5a admin admin] PATH_INFO: `/v3/projects/3df9eb441b5440f6800e5ce6ae3395b3` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:33.247370 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514fde45-b5ae-42ca-8d3f-8ee90ec16d5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3df9eb441b5440f6800e5ce6ae3395b3)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:33.250306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514fde45-b5ae-42ca-8d3f-8ee90ec16d5a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:33.521709 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-514fde45-b5ae-42ca-8d3f-8ee90ec16d5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3df9eb441b5440f6800e5ce6ae3395b3', 'request_id': 'req-514fde45-b5ae-42ca-8d3f-8ee90ec16d5a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:33.529173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-514fde45-b5ae-42ca-8d3f-8ee90ec16d5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3df9eb441b5440f6800e5ce6ae3395b3', 'request_id': 'req-514fde45-b5ae-42ca-8d3f-8ee90ec16d5a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:33.544476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-514fde45-b5ae-42ca-8d3f-8ee90ec16d5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3df9eb441b5440f6800e5ce6ae3395b3', 'request_id': 'req-514fde45-b5ae-42ca-8d3f-8ee90ec16d5a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:33.635651 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:33.645752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-514fde45-b5ae-42ca-8d3f-8ee90ec16d5a admin admin] The token cache is being invalidate because project 3df9eb441b5440f6800e5ce6ae3395b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:17:33.645752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-514fde45-b5ae-42ca-8d3f-8ee90ec16d5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:33.645752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-514fde45-b5ae-42ca-8d3f-8ee90ec16d5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:33.653795 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 855/1711] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:17:33 2022] DELETE /identity/v3/projects/3df9eb441b5440f6800e5ce6ae3395b3 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:33.656302 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388d322-5d73-4649-b687-cadee2526a30 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:33.656590 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388d322-5d73-4649-b687-cadee2526a30 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:33.656868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388d322-5d73-4649-b687-cadee2526a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:33.692162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0388d322-5d73-4649-b687-cadee2526a30 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:33.696541 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0388d322-5d73-4649-b687-cadee2526a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:33.845834 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 857/1712] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:33.855400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62d5ff-4cc7-461b-9613-2e227bb7a8f6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:33.858725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62d5ff-4cc7-461b-9613-2e227bb7a8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:33.859186 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62d5ff-4cc7-461b-9613-2e227bb7a8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:33.899742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5b62d5ff-4cc7-461b-9613-2e227bb7a8f6 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:33.909149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5b62d5ff-4cc7-461b-9613-2e227bb7a8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:33.938750 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388d322-5d73-4649-b687-cadee2526a30 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:33.944825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 856/1713] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:33.954145 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388d322-5d73-4649-b687-cadee2526a30 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:33.974393 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84880b84-8c01-4410-8a26-7f89a30debd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:33.976212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84880b84-8c01-4410-8a26-7f89a30debd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:33.978337 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84880b84-8c01-4410-8a26-7f89a30debd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:33.978736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84880b84-8c01-4410-8a26-7f89a30debd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:33.979899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84880b84-8c01-4410-8a26-7f89a30debd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:17:33.983266 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84880b84-8c01-4410-8a26-7f89a30debd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:33.988357 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84880b84-8c01-4410-8a26-7f89a30debd3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:34.027573 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 858/1714] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:17:34.032127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62d5ff-4cc7-461b-9613-2e227bb7a8f6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:34.048120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62d5ff-4cc7-461b-9613-2e227bb7a8f6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:34.058741 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff0874-263a-4506-94fa-3e9c3c14b878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:34.059729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff0874-263a-4506-94fa-3e9c3c14b878 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:34.060167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff0874-263a-4506-94fa-3e9c3c14b878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:34.060592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff0874-263a-4506-94fa-3e9c3c14b878 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:34.061756 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff0874-263a-4506-94fa-3e9c3c14b878 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:34.065221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff0874-263a-4506-94fa-3e9c3c14b878 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:34.193779 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 857/1715] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:17:34 2022] POST /identity/v3/projects => generated 371 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:34.195395 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84880b84-8c01-4410-8a26-7f89a30debd3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:34.213778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84880b84-8c01-4410-8a26-7f89a30debd3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:34.315569 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62177506-b292-4b67-93d4-864c889fd5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:34.331518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62177506-b292-4b67-93d4-864c889fd5bd admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:34.332352 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62177506-b292-4b67-93d4-864c889fd5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:34.332849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62177506-b292-4b67-93d4-864c889fd5bd admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:34.335974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62177506-b292-4b67-93d4-864c889fd5bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:34.339016 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62177506-b292-4b67-93d4-864c889fd5bd admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:34.470424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 859/1716] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:17:34 2022] POST /identity/v3/users => generated 286 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:34.474100 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff0874-263a-4506-94fa-3e9c3c14b878 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:34.491070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff0874-263a-4506-94fa-3e9c3c14b878 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:34.502728 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163a5f7-2b7d-4a81-bf5a-f259628ce88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:34.503651 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163a5f7-2b7d-4a81-bf5a-f259628ce88e admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:34.504106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163a5f7-2b7d-4a81-bf5a-f259628ce88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:34.504545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163a5f7-2b7d-4a81-bf5a-f259628ce88e admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:34.505608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163a5f7-2b7d-4a81-bf5a-f259628ce88e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:34.507427 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163a5f7-2b7d-4a81-bf5a-f259628ce88e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:34.533902 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 858/1717] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:34.538355 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62177506-b292-4b67-93d4-864c889fd5bd admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:34.559492 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62177506-b292-4b67-93d4-864c889fd5bd admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:34.575342 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10055f51-9a33-42a4-8d9c-a40146cccfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:34.576220 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10055f51-9a33-42a4-8d9c-a40146cccfbf admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:34.576328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10055f51-9a33-42a4-8d9c-a40146cccfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:34.577143 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10055f51-9a33-42a4-8d9c-a40146cccfbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:34.577885 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10055f51-9a33-42a4-8d9c-a40146cccfbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:34.579563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10055f51-9a33-42a4-8d9c-a40146cccfbf admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:34.590815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:34.617152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 860/1718] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:34.621093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163a5f7-2b7d-4a81-bf5a-f259628ce88e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:34.654603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163a5f7-2b7d-4a81-bf5a-f259628ce88e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:34.679472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455c035-6525-4a8b-a179-4aad5a080ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:34.680504 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455c035-6525-4a8b-a179-4aad5a080ac9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:34.680940 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455c035-6525-4a8b-a179-4aad5a080ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:34.681212 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455c035-6525-4a8b-a179-4aad5a080ac9 admin admin] PATH_INFO: `/v3/projects/5c13a5c860de44a4950a165e8273fad0/users/c3be6ae7a5fb415b93783dd7b0e79fe1/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:34.744615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455c035-6525-4a8b-a179-4aad5a080ac9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c13a5c860de44a4950a165e8273fad0, user_id=c3be6ae7a5fb415b93783dd7b0e79fe1, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:34.748719 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455c035-6525-4a8b-a179-4aad5a080ac9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:34.823121 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ceac2-152b-41a4-a9a6-a8c01110c110 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:34.823678 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ceac2-152b-41a4-a9a6-a8c01110c110 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:34.824044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ceac2-152b-41a4-a9a6-a8c01110c110 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:34.825876 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 861/1719] 10.222.0.16 () {54 vars in 960 bytes} [Fri Jun 24 04:17:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:17:34.832236 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a59bc0-57df-409f-917f-bd1055f6e312 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:34.832723 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a59bc0-57df-409f-917f-bd1055f6e312 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:34.833108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a59bc0-57df-409f-917f-bd1055f6e312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:34.863468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-72a59bc0-57df-409f-917f-bd1055f6e312 None None] MFA Rules not processed for user `7662e61619d44c92af3ffacbacba3913`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:34.864889 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-72a59bc0-57df-409f-917f-bd1055f6e312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:34.925982 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 859/1720] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:34 2022] PUT /identity/v3/projects/5c13a5c860de44a4950a165e8273fad0/users/c3be6ae7a5fb415b93783dd7b0e79fe1/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:34.935297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440f67d-0087-4547-a93d-a635428ca422 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:34.935623 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440f67d-0087-4547-a93d-a635428ca422 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:34.936170 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440f67d-0087-4547-a93d-a635428ca422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:34.992195 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 862/1721] 10.222.0.16 () {58 vars in 1057 bytes} [Fri Jun 24 04:17:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:17:35.024398 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a59bc0-57df-409f-917f-bd1055f6e312 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:35.052103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a59bc0-57df-409f-917f-bd1055f6e312 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:35.058033 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9440f67d-0087-4547-a93d-a635428ca422 None None] MFA Rules not processed for user `c3be6ae7a5fb415b93783dd7b0e79fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:35.069898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9440f67d-0087-4547-a93d-a635428ca422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:35.188560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 860/1722] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:17:35.191106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe56ac-01de-4eb6-8f33-8bf6414662c9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:35.191399 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe56ac-01de-4eb6-8f33-8bf6414662c9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:35.191684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe56ac-01de-4eb6-8f33-8bf6414662c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:35.226566 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f02ca-9d49-47d9-aa2c-61d4aec292fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7662e61619d44c92af3ffacbacba3913', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:35.227749 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f02ca-9d49-47d9-aa2c-61d4aec292fe service placement] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:35.227749 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f02ca-9d49-47d9-aa2c-61d4aec292fe service placement] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:35.228220 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f02ca-9d49-47d9-aa2c-61d4aec292fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:35.230177 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d11f02ca-9d49-47d9-aa2c-61d4aec292fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:35.238608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-76fe56ac-01de-4eb6-8f33-8bf6414662c9 None None] MFA Rules not processed for user `b158078d039a4979a9c0cf2be26c4b14`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:35.239892 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-76fe56ac-01de-4eb6-8f33-8bf6414662c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:35.254805 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f02ca-9d49-47d9-aa2c-61d4aec292fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:35.257782 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f02ca-9d49-47d9-aa2c-61d4aec292fe service placement] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:35.270483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:35.306612 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 863/1723] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:35.309569 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f02ca-9d49-47d9-aa2c-61d4aec292fe service placement] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:35.443229 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 861/1724] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:35.458796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e92a7f-18a0-4779-b3b0-fddc77e25d81 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:35.465127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e92a7f-18a0-4779-b3b0-fddc77e25d81 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:35.465127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e92a7f-18a0-4779-b3b0-fddc77e25d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:35.480428 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f02ca-9d49-47d9-aa2c-61d4aec292fe service placement] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:35.490216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934c8ca-3f5c-4aad-8566-7b70f190744f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:35.492474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934c8ca-3f5c-4aad-8566-7b70f190744f admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:35.494088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934c8ca-3f5c-4aad-8566-7b70f190744f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:35.494924 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934c8ca-3f5c-4aad-8566-7b70f190744f admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:35.496617 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f934c8ca-3f5c-4aad-8566-7b70f190744f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:35.501083 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f934c8ca-3f5c-4aad-8566-7b70f190744f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:35.504433 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-13e92a7f-18a0-4779-b3b0-fddc77e25d81 None None] MFA Rules not processed for user `b158078d039a4979a9c0cf2be26c4b14`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:35.507948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-13e92a7f-18a0-4779-b3b0-fddc77e25d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:35.528644 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 862/1725] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:35.543647 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e92a7f-18a0-4779-b3b0-fddc77e25d81 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:35.547342 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-13e92a7f-18a0-4779-b3b0-fddc77e25d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:35.612013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 864/1726] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:17:35 2022] POST /identity/v3/projects => generated 371 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:35.614713 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934c8ca-3f5c-4aad-8566-7b70f190744f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:35.716096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e92a7f-18a0-4779-b3b0-fddc77e25d81 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:35.730820 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934c8ca-3f5c-4aad-8566-7b70f190744f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:35.739030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb203687-8e89-4b82-ab01-8913f945e5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:35.739030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb203687-8e89-4b82-ab01-8913f945e5d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:35.739250 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb203687-8e89-4b82-ab01-8913f945e5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:35.739608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb203687-8e89-4b82-ab01-8913f945e5d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:35.740384 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb203687-8e89-4b82-ab01-8913f945e5d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:35.742741 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb203687-8e89-4b82-ab01-8913f945e5d1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:35.869143 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 865/1727] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:17:35 2022] POST /identity/v3/users => generated 292 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:35.870840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ed158-3974-4f37-b247-ce4a458531ea None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:35.871160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ed158-3974-4f37-b247-ce4a458531ea None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:35.871471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3ed158-3974-4f37-b247-ce4a458531ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:35.878962 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd78c85-c188-4dac-a5cd-8e03aa700fa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:35.879639 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd78c85-c188-4dac-a5cd-8e03aa700fa0 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:35.879927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd78c85-c188-4dac-a5cd-8e03aa700fa0 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:35.880212 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd78c85-c188-4dac-a5cd-8e03aa700fa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:35.936281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ca3ed158-3974-4f37-b247-ce4a458531ea None None] MFA Rules not processed for user `066d1521db464b4996f6daf6548dcc38`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:35.960174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ca3ed158-3974-4f37-b247-ce4a458531ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:36.011872 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd78c85-c188-4dac-a5cd-8e03aa700fa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:36.035291 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd78c85-c188-4dac-a5cd-8e03aa700fa0 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:36.063028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:36.074219 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 863/1728] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:17:36.074219 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd78c85-c188-4dac-a5cd-8e03aa700fa0 service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:36.092104 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd78c85-c188-4dac-a5cd-8e03aa700fa0 service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:36.113851 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07166a7d-f1a8-4190-9366-e3a547a322c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:36.115333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07166a7d-f1a8-4190-9366-e3a547a322c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:36.118515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07166a7d-f1a8-4190-9366-e3a547a322c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:36.120462 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07166a7d-f1a8-4190-9366-e3a547a322c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:36.125787 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07166a7d-f1a8-4190-9366-e3a547a322c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:36.136691 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07166a7d-f1a8-4190-9366-e3a547a322c0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:36.155246 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 864/1729] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:36.156766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a4e6c-4d07-451d-89ff-be66e1c113ec None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:36.157139 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a4e6c-4d07-451d-89ff-be66e1c113ec None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:36.157514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a4e6c-4d07-451d-89ff-be66e1c113ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:36.179991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 866/1730] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:36.179991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ed158-3974-4f37-b247-ce4a458531ea None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:36.213481 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3ed158-3974-4f37-b247-ce4a458531ea None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:36.218928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf1148-d3df-4937-9ca9-2cd420377f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:36.219487 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-aa2a4e6c-4d07-451d-89ff-be66e1c113ec None None] MFA Rules not processed for user `9ffa029a3b1e4919b00e58a9737816d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:36.220610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf1148-d3df-4937-9ca9-2cd420377f80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:36.220980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf1148-d3df-4937-9ca9-2cd420377f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:36.221312 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-aa2a4e6c-4d07-451d-89ff-be66e1c113ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:36.221433 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf1148-d3df-4937-9ca9-2cd420377f80 admin admin] PATH_INFO: `/v3/projects/e741564bb69147b6b865ba5c5c5465f0/users/a284b408e06c4737bc0098afe08f74e5/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:36.261917 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf1148-d3df-4937-9ca9-2cd420377f80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e741564bb69147b6b865ba5c5c5465f0, user_id=a284b408e06c4737bc0098afe08f74e5, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:36.273033 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf1148-d3df-4937-9ca9-2cd420377f80 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:36.371269 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 865/1731] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:36.371269 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a4e6c-4d07-451d-89ff-be66e1c113ec None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:36.371269 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-aa2a4e6c-4d07-451d-89ff-be66e1c113ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:36.411650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a4e6c-4d07-451d-89ff-be66e1c113ec None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:36.419196 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ddc64-c018-419e-ba7c-3de8499bf9f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:36.421578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ddc64-c018-419e-ba7c-3de8499bf9f2 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:36.421578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ddc64-c018-419e-ba7c-3de8499bf9f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:36.421578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90ddc64-c018-419e-ba7c-3de8499bf9f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:36.428216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 867/1732] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:36 2022] PUT /identity/v3/projects/e741564bb69147b6b865ba5c5c5465f0/users/a284b408e06c4737bc0098afe08f74e5/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:36.429145 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf1148-d3df-4937-9ca9-2cd420377f80 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:36.433775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ddc64-c018-419e-ba7c-3de8499bf9f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:36.436392 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90ddc64-c018-419e-ba7c-3de8499bf9f2 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:36.463872 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 866/1733] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:36.563170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf1148-d3df-4937-9ca9-2cd420377f80 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:36.575907 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdc0d5-390d-4900-8344-530f0c1d7b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:36.575907 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdc0d5-390d-4900-8344-530f0c1d7b70 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:36.577082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdc0d5-390d-4900-8344-530f0c1d7b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:36.577687 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdc0d5-390d-4900-8344-530f0c1d7b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:36.578827 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbdc0d5-390d-4900-8344-530f0c1d7b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:36.584798 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbdc0d5-390d-4900-8344-530f0c1d7b70 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:36.603704 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 868/1734] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:36.607945 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ddc64-c018-419e-ba7c-3de8499bf9f2 service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:36.650359 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90ddc64-c018-419e-ba7c-3de8499bf9f2 service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:36.662996 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af057aed-c31f-479e-a5d7-c69e4ee8028d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:36.664143 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af057aed-c31f-479e-a5d7-c69e4ee8028d admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:36.664558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af057aed-c31f-479e-a5d7-c69e4ee8028d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:36.665024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af057aed-c31f-479e-a5d7-c69e4ee8028d admin admin] PATH_INFO: `/v3/projects/e741564bb69147b6b865ba5c5c5465f0/users/a284b408e06c4737bc0098afe08f74e5/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:36.669650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af057aed-c31f-479e-a5d7-c69e4ee8028d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e741564bb69147b6b865ba5c5c5465f0, user_id=a284b408e06c4737bc0098afe08f74e5, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:36.675756 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af057aed-c31f-479e-a5d7-c69e4ee8028d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:36.678985 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:36.706053 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 867/1735] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:36 2022] PUT /identity/v3/projects/e741564bb69147b6b865ba5c5c5465f0/users/a284b408e06c4737bc0098afe08f74e5/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:36.707946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df325f-935f-4953-8871-b7ee017cda38 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:36.708465 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df325f-935f-4953-8871-b7ee017cda38 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:36.708842 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6df325f-935f-4953-8871-b7ee017cda38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:36.768406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c6df325f-935f-4953-8871-b7ee017cda38 None None] MFA Rules not processed for user `a284b408e06c4737bc0098afe08f74e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:36.768406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c6df325f-935f-4953-8871-b7ee017cda38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:36.925415 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 869/1736] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:36.937597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af057aed-c31f-479e-a5d7-c69e4ee8028d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:36.939805 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-af057aed-c31f-479e-a5d7-c69e4ee8028d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:37.183741 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af057aed-c31f-479e-a5d7-c69e4ee8028d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:37.207939 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3f167-7066-4cfb-ac9c-9c86f4b6a58c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:37.216814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3f167-7066-4cfb-ac9c-9c86f4b6a58c service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:37.219009 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3f167-7066-4cfb-ac9c-9c86f4b6a58c service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:37.220090 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3f167-7066-4cfb-ac9c-9c86f4b6a58c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:37.278116 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3f167-7066-4cfb-ac9c-9c86f4b6a58c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:37.284309 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b3f167-7066-4cfb-ac9c-9c86f4b6a58c service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:37.318304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 868/1737] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:36 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:37.637608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b93d57-048c-4c43-9cbd-a22a6dc11f81 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:37.638332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b93d57-048c-4c43-9cbd-a22a6dc11f81 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:37.639474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b93d57-048c-4c43-9cbd-a22a6dc11f81 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:37.641579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 870/1738] 10.222.0.16 () {54 vars in 912 bytes} [Fri Jun 24 04:17:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:17:37.656106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508ffca-e990-4cb0-8200-c72a10b04619 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:37.656648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508ffca-e990-4cb0-8200-c72a10b04619 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:37.657179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508ffca-e990-4cb0-8200-c72a10b04619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:37.662852 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b93d57-048c-4c43-9cbd-a22a6dc11f81 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:37.667236 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-15b93d57-048c-4c43-9cbd-a22a6dc11f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:37.704720 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b93d57-048c-4c43-9cbd-a22a6dc11f81 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:37.705730 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2508ffca-e990-4cb0-8200-c72a10b04619 None None] MFA Rules not processed for user `fd98b54f587f4b42b4dd0a058b30c684`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:37.708268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2508ffca-e990-4cb0-8200-c72a10b04619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:37.712410 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eef84f-99de-4cf3-a2d5-f37353257039 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:37.713214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eef84f-99de-4cf3-a2d5-f37353257039 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:37.713629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eef84f-99de-4cf3-a2d5-f37353257039 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:37.714028 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eef84f-99de-4cf3-a2d5-f37353257039 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:37.832089 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eef84f-99de-4cf3-a2d5-f37353257039 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:37.833719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eef84f-99de-4cf3-a2d5-f37353257039 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:37.870428 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 871/1739] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:37.907686 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 869/1740] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jun 24 04:17:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:37.910570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:37.972067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eef84f-99de-4cf3-a2d5-f37353257039 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:37.974341 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-61eef84f-99de-4cf3-a2d5-f37353257039 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:37.991820 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eef84f-99de-4cf3-a2d5-f37353257039 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:38.002100 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc90ad-3120-49d6-a8f3-5f2a041809fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:38.009808 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc90ad-3120-49d6-a8f3-5f2a041809fc service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:38.009808 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc90ad-3120-49d6-a8f3-5f2a041809fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:38.009808 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc90ad-3120-49d6-a8f3-5f2a041809fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:38.020690 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc90ad-3120-49d6-a8f3-5f2a041809fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:38.022818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc90ad-3120-49d6-a8f3-5f2a041809fc service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:38.032525 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:38.045154 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 872/1741] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:38.143728 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f02c69-add5-48ee-aa70-d52c1179d826 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:38.144324 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f02c69-add5-48ee-aa70-d52c1179d826 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:38.144419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f02c69-add5-48ee-aa70-d52c1179d826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:38.195749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-34f02c69-add5-48ee-aa70-d52c1179d826 None None] MFA Rules not processed for user `6d53a9bd8dd8494e98bf7bd730bac2e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:38.197116 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-34f02c69-add5-48ee-aa70-d52c1179d826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:38.364791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 870/1742] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:38 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:38.369642 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc90ad-3120-49d6-a8f3-5f2a041809fc service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:38.372739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7ebc90ad-3120-49d6-a8f3-5f2a041809fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:38.510258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc90ad-3120-49d6-a8f3-5f2a041809fc service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:38.516887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2800f-cdff-4b20-8a64-109358719558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:38.518411 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2800f-cdff-4b20-8a64-109358719558 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:38.518876 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2800f-cdff-4b20-8a64-109358719558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:38.519310 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2800f-cdff-4b20-8a64-109358719558 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:38.520608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2800f-cdff-4b20-8a64-109358719558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:38.523455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2800f-cdff-4b20-8a64-109358719558 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:38.716595 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 873/1743] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:17:38 2022] POST /identity/v3/projects => generated 403 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:38.718424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f02c69-add5-48ee-aa70-d52c1179d826 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:38.737802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f02c69-add5-48ee-aa70-d52c1179d826 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:38.960393 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40a2719-d2ca-4541-91d8-59435896dce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:38.961651 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a2719-d2ca-4541-91d8-59435896dce2 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:38.962140 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a2719-d2ca-4541-91d8-59435896dce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:38.967464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40a2719-d2ca-4541-91d8-59435896dce2 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:38.967464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a2719-d2ca-4541-91d8-59435896dce2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:38.967464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40a2719-d2ca-4541-91d8-59435896dce2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:39.072443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 871/1744] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:17:38 2022] POST /identity/v3/users => generated 308 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:17:39.077076 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2800f-cdff-4b20-8a64-109358719558 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:39.098308 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:39.103700 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2800f-cdff-4b20-8a64-109358719558 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:39.137504 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a751f-6101-4573-8531-d483193f8ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:39.138223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a751f-6101-4573-8531-d483193f8ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:39.142653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a751f-6101-4573-8531-d483193f8ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:39.143047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a751f-6101-4573-8531-d483193f8ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:39.143836 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a751f-6101-4573-8531-d483193f8ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:39.145509 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a751f-6101-4573-8531-d483193f8ff3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:39.189275 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70f666-0e67-4318-a9ec-b86cffb1c91d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:39.189572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70f666-0e67-4318-a9ec-b86cffb1c91d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:39.194090 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70f666-0e67-4318-a9ec-b86cffb1c91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:39.230033 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 874/1745] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:39.243063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a751f-6101-4573-8531-d483193f8ff3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:39.251733 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6c70f666-0e67-4318-a9ec-b86cffb1c91d None None] MFA Rules not processed for user `b158078d039a4979a9c0cf2be26c4b14`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:39.252560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6c70f666-0e67-4318-a9ec-b86cffb1c91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:39.294067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a751f-6101-4573-8531-d483193f8ff3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:39.312489 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1848714-8858-4bc6-8a53-1a818016eb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:39.313794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1848714-8858-4bc6-8a53-1a818016eb45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:39.314394 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1848714-8858-4bc6-8a53-1a818016eb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:39.314957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1848714-8858-4bc6-8a53-1a818016eb45 admin admin] PATH_INFO: `/v3/projects/8ad2d8b967314ba19b4bde93af3822ac/users/24e7571a6f1647baac1197edeaff3692/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:39.351129 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1848714-8858-4bc6-8a53-1a818016eb45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ad2d8b967314ba19b4bde93af3822ac, user_id=24e7571a6f1647baac1197edeaff3692, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:39.355920 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1848714-8858-4bc6-8a53-1a818016eb45 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:39.461587 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 875/1746] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:39 2022] PUT /identity/v3/projects/8ad2d8b967314ba19b4bde93af3822ac/users/24e7571a6f1647baac1197edeaff3692/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:39.465527 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1848714-8858-4bc6-8a53-1a818016eb45 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:39.470416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 872/1747] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:39.693144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1848714-8858-4bc6-8a53-1a818016eb45 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:39.711144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ab8cb-7483-44d6-b569-4f6dd2533d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:39.712008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ab8cb-7483-44d6-b569-4f6dd2533d87 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:39.715141 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ab8cb-7483-44d6-b569-4f6dd2533d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:39.715489 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ab8cb-7483-44d6-b569-4f6dd2533d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:39.720184 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ab8cb-7483-44d6-b569-4f6dd2533d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:39.729544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ab8cb-7483-44d6-b569-4f6dd2533d87 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:39.801609 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:39.808735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70f666-0e67-4318-a9ec-b86cffb1c91d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:39.811732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 876/1748] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:39.850954 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70f666-0e67-4318-a9ec-b86cffb1c91d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:39.881135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188c554-f0cc-4a96-a15f-c346d236f653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:39.888247 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188c554-f0cc-4a96-a15f-c346d236f653 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:39.891170 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188c554-f0cc-4a96-a15f-c346d236f653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:39.893759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188c554-f0cc-4a96-a15f-c346d236f653 admin admin] PATH_INFO: `/v3/projects/8ad2d8b967314ba19b4bde93af3822ac/users/24e7571a6f1647baac1197edeaff3692/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:39.909660 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6188c554-f0cc-4a96-a15f-c346d236f653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ad2d8b967314ba19b4bde93af3822ac, user_id=24e7571a6f1647baac1197edeaff3692, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:39.919517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6188c554-f0cc-4a96-a15f-c346d236f653 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:39.967620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 873/1749] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:39 2022] PUT /identity/v3/projects/8ad2d8b967314ba19b4bde93af3822ac/users/24e7571a6f1647baac1197edeaff3692/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:41.030140 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486380b3-bc25-4218-8b8f-50df8864e92a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:41.034254 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486380b3-bc25-4218-8b8f-50df8864e92a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:41.035282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486380b3-bc25-4218-8b8f-50df8864e92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:41.076017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-486380b3-bc25-4218-8b8f-50df8864e92a None None] MFA Rules not processed for user `114d227e9645409e9082cf9b09b05c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:41.076017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-486380b3-bc25-4218-8b8f-50df8864e92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:41.166357 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 877/1750] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:41.583626 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38eaed-8414-4871-a18a-c858c0fe17f7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:41.584153 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38eaed-8414-4871-a18a-c858c0fe17f7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:41.584153 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38eaed-8414-4871-a18a-c858c0fe17f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:41.591829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:41.607599 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-fe38eaed-8414-4871-a18a-c858c0fe17f7 None None] MFA Rules not processed for user `98f58f91d17949198e66883d234a9e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:41.609388 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fe38eaed-8414-4871-a18a-c858c0fe17f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:41.796080 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 874/1751] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jun 24 04:17:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:17:42.449078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486380b3-bc25-4218-8b8f-50df8864e92a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:42.451426 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-486380b3-bc25-4218-8b8f-50df8864e92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:42.681100 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486380b3-bc25-4218-8b8f-50df8864e92a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:42.688843 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a273e-1336-46ec-b445-cf9ba486c48e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8e1f1e3c7e42cc9fa6f82aaac55129', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:42.690186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a273e-1336-46ec-b445-cf9ba486c48e service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:42.691582 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a273e-1336-46ec-b445-cf9ba486c48e service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:42.692308 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a273e-1336-46ec-b445-cf9ba486c48e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:42.702969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:42.708604 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a273e-1336-46ec-b445-cf9ba486c48e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:42.711071 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4a273e-1336-46ec-b445-cf9ba486c48e service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:42.724396 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 878/1752] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jun 24 04:17:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 04:17:43.436440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187261d4-dfbe-420f-9fad-b33ebf39ce13 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:43.437094 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187261d4-dfbe-420f-9fad-b33ebf39ce13 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:43.437094 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187261d4-dfbe-420f-9fad-b33ebf39ce13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:43.503788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-187261d4-dfbe-420f-9fad-b33ebf39ce13 None None] MFA Rules not processed for user `066d1521db464b4996f6daf6548dcc38`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:43.505040 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-187261d4-dfbe-420f-9fad-b33ebf39ce13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:43.626602 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 875/1753] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:43.962698 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6e46f-5a6c-4987-bbb5-fde23cb61168 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:43.963905 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6e46f-5a6c-4987-bbb5-fde23cb61168 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:43.964629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6e46f-5a6c-4987-bbb5-fde23cb61168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:44.041664 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-48f6e46f-5a6c-4987-bbb5-fde23cb61168 None None] MFA Rules not processed for user `24e7571a6f1647baac1197edeaff3692`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:44.042934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-48f6e46f-5a6c-4987-bbb5-fde23cb61168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:44.214029 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 879/1754] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:43 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:45.546888 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db92d90-9e83-460d-be8a-a69e9492cb66 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:45.547459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db92d90-9e83-460d-be8a-a69e9492cb66 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:45.547459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db92d90-9e83-460d-be8a-a69e9492cb66 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:45.548944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 876/1755] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jun 24 04:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:17:45.564474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795eab78-25c9-4d7f-80bd-9f4dc0b43023 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:45.564992 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795eab78-25c9-4d7f-80bd-9f4dc0b43023 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:45.565283 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795eab78-25c9-4d7f-80bd-9f4dc0b43023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:45.591370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-795eab78-25c9-4d7f-80bd-9f4dc0b43023 None None] MFA Rules not processed for user `7662e61619d44c92af3ffacbacba3913`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:45.593622 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-795eab78-25c9-4d7f-80bd-9f4dc0b43023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:45.668608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 880/1756] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jun 24 04:17:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:45.681499 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db92d90-9e83-460d-be8a-a69e9492cb66 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:45.683326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6db92d90-9e83-460d-be8a-a69e9492cb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:45.700819 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db92d90-9e83-460d-be8a-a69e9492cb66 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:45.707154 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafda9b5-64b1-47d3-9264-95dd6717cacb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7662e61619d44c92af3ffacbacba3913', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:45.707990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafda9b5-64b1-47d3-9264-95dd6717cacb service placement] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:45.708574 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafda9b5-64b1-47d3-9264-95dd6717cacb service placement] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:45.709378 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafda9b5-64b1-47d3-9264-95dd6717cacb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:45.720880 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafda9b5-64b1-47d3-9264-95dd6717cacb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:45.722715 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafda9b5-64b1-47d3-9264-95dd6717cacb service placement] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:45.736859 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 877/1757] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:46.463392 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784899b-34d3-4334-8ad9-67e4284332bf None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:46.463991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784899b-34d3-4334-8ad9-67e4284332bf None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:46.464443 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784899b-34d3-4334-8ad9-67e4284332bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:46.537252 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-3784899b-34d3-4334-8ad9-67e4284332bf None None] MFA Rules not processed for user `fd98b54f587f4b42b4dd0a058b30c684`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:46.538701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3784899b-34d3-4334-8ad9-67e4284332bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:46.690357 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 881/1758] 10.222.0.43 () {58 vars in 1015 bytes} [Fri Jun 24 04:17:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:17:46.754433 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafda9b5-64b1-47d3-9264-95dd6717cacb service placement] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:46.767864 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafda9b5-64b1-47d3-9264-95dd6717cacb service placement] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:46.772546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012d33e-c17d-4b2d-a405-a05e692a6f28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:46.773480 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012d33e-c17d-4b2d-a405-a05e692a6f28 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:46.773994 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012d33e-c17d-4b2d-a405-a05e692a6f28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:46.774529 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012d33e-c17d-4b2d-a405-a05e692a6f28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:46.792691 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012d33e-c17d-4b2d-a405-a05e692a6f28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:46.794478 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012d33e-c17d-4b2d-a405-a05e692a6f28 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:46.823132 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 878/1759] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:48.149465 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784899b-34d3-4334-8ad9-67e4284332bf None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:48.253734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:48.454278 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784899b-34d3-4334-8ad9-67e4284332bf None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:48.499672 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15779f9-e169-4a38-8608-bddb5769cac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:48.500591 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15779f9-e169-4a38-8608-bddb5769cac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:48.500880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15779f9-e169-4a38-8608-bddb5769cac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:48.501162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15779f9-e169-4a38-8608-bddb5769cac2 admin admin] PATH_INFO: `/v3/users/6d53a9bd8dd8494e98bf7bd730bac2e2` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:48.507836 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15779f9-e169-4a38-8608-bddb5769cac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d53a9bd8dd8494e98bf7bd730bac2e2)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:48.518444 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15779f9-e169-4a38-8608-bddb5769cac2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:48.873460 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a15779f9-e169-4a38-8608-bddb5769cac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d53a9bd8dd8494e98bf7bd730bac2e2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:48.934738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a15779f9-e169-4a38-8608-bddb5769cac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d53a9bd8dd8494e98bf7bd730bac2e2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:49.054315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c49a1-bff1-49fe-824c-886935dced4e None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:49.055309 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c49a1-bff1-49fe-824c-886935dced4e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:49.055751 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c49a1-bff1-49fe-824c-886935dced4e None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:49.057526 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 879/1760] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jun 24 04:17:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 04:17:49.061212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a15779f9-e169-4a38-8608-bddb5769cac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d53a9bd8dd8494e98bf7bd730bac2e2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:49.062617 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a15779f9-e169-4a38-8608-bddb5769cac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d53a9bd8dd8494e98bf7bd730bac2e2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:49.068776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a15779f9-e169-4a38-8608-bddb5769cac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d53a9bd8dd8494e98bf7bd730bac2e2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:49.081859 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceff7f5-eed7-41b0-aeef-773bc8ac96ae None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:49.082452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceff7f5-eed7-41b0-aeef-773bc8ac96ae None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:49.086221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceff7f5-eed7-41b0-aeef-773bc8ac96ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:49.131824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3ceff7f5-eed7-41b0-aeef-773bc8ac96ae None None] MFA Rules not processed for user `80b7b3ec60db4de19bbf7d08f111f470`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:49.133307 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3ceff7f5-eed7-41b0-aeef-773bc8ac96ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:49.148480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 882/1761] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:17:48 2022] DELETE /identity/v3/users/6d53a9bd8dd8494e98bf7bd730bac2e2 => generated 0 bytes in 995 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:49.387863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 880/1762] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jun 24 04:17:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:50.334287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15779f9-e169-4a38-8608-bddb5769cac2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:50.336744 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a15779f9-e169-4a38-8608-bddb5769cac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:50.354162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15779f9-e169-4a38-8608-bddb5769cac2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:50.363115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367d5a0-79f3-4c44-98b8-9756b38b7623 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:50.364807 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367d5a0-79f3-4c44-98b8-9756b38b7623 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:50.365603 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367d5a0-79f3-4c44-98b8-9756b38b7623 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:50.366504 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367d5a0-79f3-4c44-98b8-9756b38b7623 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:50.378794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367d5a0-79f3-4c44-98b8-9756b38b7623 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:50.380853 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3367d5a0-79f3-4c44-98b8-9756b38b7623 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:50.407762 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 883/1763] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:51.016287 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceff7f5-eed7-41b0-aeef-773bc8ac96ae None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:51.019604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3ceff7f5-eed7-41b0-aeef-773bc8ac96ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:51.079968 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:51.185949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceff7f5-eed7-41b0-aeef-773bc8ac96ae None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:51.208386 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72c323-ea82-4441-b34d-6ee4f8dfddc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:51.209575 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72c323-ea82-4441-b34d-6ee4f8dfddc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:51.210031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72c323-ea82-4441-b34d-6ee4f8dfddc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:51.210503 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b72c323-ea82-4441-b34d-6ee4f8dfddc0 admin admin] PATH_INFO: `/v3/projects/f414fbc699c141fb9990984e06e90155` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:51.211988 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72c323-ea82-4441-b34d-6ee4f8dfddc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f414fbc699c141fb9990984e06e90155)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:51.215418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b72c323-ea82-4441-b34d-6ee4f8dfddc0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:51.396159 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0b72c323-ea82-4441-b34d-6ee4f8dfddc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f414fbc699c141fb9990984e06e90155', 'request_id': 'req-0b72c323-ea82-4441-b34d-6ee4f8dfddc0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:51.406245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0b72c323-ea82-4441-b34d-6ee4f8dfddc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f414fbc699c141fb9990984e06e90155', 'request_id': 'req-0b72c323-ea82-4441-b34d-6ee4f8dfddc0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:51.415013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0b72c323-ea82-4441-b34d-6ee4f8dfddc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f414fbc699c141fb9990984e06e90155', 'request_id': 'req-0b72c323-ea82-4441-b34d-6ee4f8dfddc0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:51.526603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0b72c323-ea82-4441-b34d-6ee4f8dfddc0 admin admin] The token cache is being invalidate because project f414fbc699c141fb9990984e06e90155 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:17:51.527022 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0b72c323-ea82-4441-b34d-6ee4f8dfddc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:51.528281 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0b72c323-ea82-4441-b34d-6ee4f8dfddc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:51.537844 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367d5a0-79f3-4c44-98b8-9756b38b7623 service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:51.539568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 881/1764] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:17:51 2022] DELETE /identity/v3/projects/f414fbc699c141fb9990984e06e90155 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:51.568232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367d5a0-79f3-4c44-98b8-9756b38b7623 service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:51.737652 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92055c25-3e42-41e1-8e3c-95e67b9f7894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:51.739682 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92055c25-3e42-41e1-8e3c-95e67b9f7894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:51.740072 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92055c25-3e42-41e1-8e3c-95e67b9f7894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:51.741651 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92055c25-3e42-41e1-8e3c-95e67b9f7894 admin admin] PATH_INFO: `/v3/users/24e7571a6f1647baac1197edeaff3692` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:51.744189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92055c25-3e42-41e1-8e3c-95e67b9f7894 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24e7571a6f1647baac1197edeaff3692)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:51.754577 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92055c25-3e42-41e1-8e3c-95e67b9f7894 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:51.976404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:52.007430 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-92055c25-3e42-41e1-8e3c-95e67b9f7894 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24e7571a6f1647baac1197edeaff3692'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:52.074092 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-92055c25-3e42-41e1-8e3c-95e67b9f7894 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24e7571a6f1647baac1197edeaff3692'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:52.183519 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-92055c25-3e42-41e1-8e3c-95e67b9f7894 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24e7571a6f1647baac1197edeaff3692'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:52.186821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-92055c25-3e42-41e1-8e3c-95e67b9f7894 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24e7571a6f1647baac1197edeaff3692'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:52.187711 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-92055c25-3e42-41e1-8e3c-95e67b9f7894 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24e7571a6f1647baac1197edeaff3692'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:52.258687 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 884/1765] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:17:51 2022] DELETE /identity/v3/users/24e7571a6f1647baac1197edeaff3692 => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:53.991778 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72c323-ea82-4441-b34d-6ee4f8dfddc0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:53.995342 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0b72c323-ea82-4441-b34d-6ee4f8dfddc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:54.037578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b72c323-ea82-4441-b34d-6ee4f8dfddc0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:54.164586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbbb4d1-62f4-4b8b-af66-869f4ee06c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:54.165773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbb4d1-62f4-4b8b-af66-869f4ee06c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:54.166259 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbb4d1-62f4-4b8b-af66-869f4ee06c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:54.166839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbbb4d1-62f4-4b8b-af66-869f4ee06c57 admin admin] PATH_INFO: `/v3/projects/8ad2d8b967314ba19b4bde93af3822ac` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:54.168565 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbbb4d1-62f4-4b8b-af66-869f4ee06c57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ad2d8b967314ba19b4bde93af3822ac)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:54.172074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbbb4d1-62f4-4b8b-af66-869f4ee06c57 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:54.397277 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1cbbb4d1-62f4-4b8b-af66-869f4ee06c57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ad2d8b967314ba19b4bde93af3822ac', 'request_id': 'req-1cbbb4d1-62f4-4b8b-af66-869f4ee06c57'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:54.402856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:54.417677 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1cbbb4d1-62f4-4b8b-af66-869f4ee06c57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ad2d8b967314ba19b4bde93af3822ac', 'request_id': 'req-1cbbb4d1-62f4-4b8b-af66-869f4ee06c57'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:54.434868 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1cbbb4d1-62f4-4b8b-af66-869f4ee06c57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ad2d8b967314ba19b4bde93af3822ac', 'request_id': 'req-1cbbb4d1-62f4-4b8b-af66-869f4ee06c57'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:54.544490 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1cbbb4d1-62f4-4b8b-af66-869f4ee06c57 admin admin] The token cache is being invalidate because project 8ad2d8b967314ba19b4bde93af3822ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:17:54.544959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1cbbb4d1-62f4-4b8b-af66-869f4ee06c57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:54.545757 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1cbbb4d1-62f4-4b8b-af66-869f4ee06c57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:54.549960 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 882/1766] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:17:53 2022] DELETE /identity/v3/projects/8ad2d8b967314ba19b4bde93af3822ac => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:54.555436 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef04065-5fbb-4454-abe7-6058c7e19eeb None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:54.556033 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef04065-5fbb-4454-abe7-6058c7e19eeb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:54.556033 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef04065-5fbb-4454-abe7-6058c7e19eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:54.596602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-eef04065-5fbb-4454-abe7-6058c7e19eeb None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:54.602978 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-eef04065-5fbb-4454-abe7-6058c7e19eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:54.713596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 885/1767] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:54.725655 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec5ab0-914f-45b5-9cff-300289abdd90 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:54.726162 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec5ab0-914f-45b5-9cff-300289abdd90 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:54.726606 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec5ab0-914f-45b5-9cff-300289abdd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:54.768565 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-16ec5ab0-914f-45b5-9cff-300289abdd90 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:54.771160 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-16ec5ab0-914f-45b5-9cff-300289abdd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:54.797446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 883/1768] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:54.800837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef04065-5fbb-4454-abe7-6058c7e19eeb None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:54.815736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef04065-5fbb-4454-abe7-6058c7e19eeb None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:54.830123 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ae8c7-fd62-433c-9c3f-a1f51cce8b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:54.837779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ae8c7-fd62-433c-9c3f-a1f51cce8b19 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:54.837779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ae8c7-fd62-433c-9c3f-a1f51cce8b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:54.837779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ae8c7-fd62-433c-9c3f-a1f51cce8b19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:54.837779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ae8c7-fd62-433c-9c3f-a1f51cce8b19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:17:54.837779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ae8c7-fd62-433c-9c3f-a1f51cce8b19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:54.837779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ae8c7-fd62-433c-9c3f-a1f51cce8b19 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:54.862173 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 886/1769] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:17:54.871871 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec5ab0-914f-45b5-9cff-300289abdd90 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:54.891351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec5ab0-914f-45b5-9cff-300289abdd90 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:54.906046 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419cc1ac-6cd7-471b-b677-ae2ee5cf1193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:54.906311 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419cc1ac-6cd7-471b-b677-ae2ee5cf1193 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:54.906901 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419cc1ac-6cd7-471b-b677-ae2ee5cf1193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:54.910503 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419cc1ac-6cd7-471b-b677-ae2ee5cf1193 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:54.911443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419cc1ac-6cd7-471b-b677-ae2ee5cf1193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:54.916256 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419cc1ac-6cd7-471b-b677-ae2ee5cf1193 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:55.148485 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 884/1770] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:17:54 2022] POST /identity/v3/projects => generated 381 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:55.152432 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ae8c7-fd62-433c-9c3f-a1f51cce8b19 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:55.173348 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ae8c7-fd62-433c-9c3f-a1f51cce8b19 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:55.384996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2928e-5824-409b-8362-6aa2533d2f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:55.386115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2928e-5824-409b-8362-6aa2533d2f73 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:55.386591 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2928e-5824-409b-8362-6aa2533d2f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:55.387155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2928e-5824-409b-8362-6aa2533d2f73 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:55.388902 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2928e-5824-409b-8362-6aa2533d2f73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:55.391307 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef2928e-5824-409b-8362-6aa2533d2f73 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:55.513940 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 887/1771] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:17:55 2022] POST /identity/v3/users => generated 291 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:55.519278 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419cc1ac-6cd7-471b-b677-ae2ee5cf1193 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:55.530916 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2928e-5824-409b-8362-6aa2533d2f73 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:55.535167 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1ef2928e-5824-409b-8362-6aa2533d2f73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:55.544011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419cc1ac-6cd7-471b-b677-ae2ee5cf1193 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:55.565561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b75b50-b894-4086-a431-ebe4749ac8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:55.568903 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b75b50-b894-4086-a431-ebe4749ac8b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:55.568903 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b75b50-b894-4086-a431-ebe4749ac8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:55.570777 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b75b50-b894-4086-a431-ebe4749ac8b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:55.571613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b75b50-b894-4086-a431-ebe4749ac8b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:55.573546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b75b50-b894-4086-a431-ebe4749ac8b8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:55.582400 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef2928e-5824-409b-8362-6aa2533d2f73 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:55.600785 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 885/1772] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:55.604555 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc2269-b8a1-4c67-a18e-918472a3ea7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:55.605232 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b75b50-b894-4086-a431-ebe4749ac8b8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:55.610043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc2269-b8a1-4c67-a18e-918472a3ea7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:55.617318 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc2269-b8a1-4c67-a18e-918472a3ea7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:55.617318 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc2269-b8a1-4c67-a18e-918472a3ea7b admin admin] PATH_INFO: `/v3/users/942e1e702362438384e98e0474353023` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:55.619335 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b75b50-b894-4086-a431-ebe4749ac8b8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:55.629878 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc2269-b8a1-4c67-a18e-918472a3ea7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=942e1e702362438384e98e0474353023)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:55.633227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc2269-b8a1-4c67-a18e-918472a3ea7b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:55.637894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915275e8-aaa4-4e38-856f-9713a9109b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:55.639365 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915275e8-aaa4-4e38-856f-9713a9109b8a admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:55.639727 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915275e8-aaa4-4e38-856f-9713a9109b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:55.640021 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915275e8-aaa4-4e38-856f-9713a9109b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:55.641305 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915275e8-aaa4-4e38-856f-9713a9109b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:55.644238 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915275e8-aaa4-4e38-856f-9713a9109b8a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:55.670621 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 886/1773] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:17:55.672204 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915275e8-aaa4-4e38-856f-9713a9109b8a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:55.694414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915275e8-aaa4-4e38-856f-9713a9109b8a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:55.720868 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678b010-fd3d-4767-9205-04b63a29fe63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:55.721835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678b010-fd3d-4767-9205-04b63a29fe63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:55.722050 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678b010-fd3d-4767-9205-04b63a29fe63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:55.722512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678b010-fd3d-4767-9205-04b63a29fe63 admin admin] PATH_INFO: `/v3/projects/cff4c8b1bc24457181baf5a594b1b8fc/users/a2b315c8801943118874ea6240bb8d1c/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:55.728145 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:55.761681 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678b010-fd3d-4767-9205-04b63a29fe63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cff4c8b1bc24457181baf5a594b1b8fc, user_id=a2b315c8801943118874ea6240bb8d1c, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:55.767703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678b010-fd3d-4767-9205-04b63a29fe63 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:55.930451 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 887/1774] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:55 2022] PUT /identity/v3/projects/cff4c8b1bc24457181baf5a594b1b8fc/users/a2b315c8801943118874ea6240bb8d1c/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:17:55.941471 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6cdc6-9868-446c-946f-795676ec58e6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:55.941863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6cdc6-9868-446c-946f-795676ec58e6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:55.942318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6cdc6-9868-446c-946f-795676ec58e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:56.011020 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7ba6cdc6-9868-446c-946f-795676ec58e6 None None] MFA Rules not processed for user `a2b315c8801943118874ea6240bb8d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:56.012239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7ba6cdc6-9868-446c-946f-795676ec58e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:56.024328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f9cc2269-b8a1-4c67-a18e-918472a3ea7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '942e1e702362438384e98e0474353023'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:56.079106 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f9cc2269-b8a1-4c67-a18e-918472a3ea7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '942e1e702362438384e98e0474353023'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:56.172504 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f9cc2269-b8a1-4c67-a18e-918472a3ea7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '942e1e702362438384e98e0474353023'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:56.173355 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f9cc2269-b8a1-4c67-a18e-918472a3ea7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '942e1e702362438384e98e0474353023'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:56.174097 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f9cc2269-b8a1-4c67-a18e-918472a3ea7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '942e1e702362438384e98e0474353023'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:56.208824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 888/1775] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:56.210084 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6cdc6-9868-446c-946f-795676ec58e6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:56.212949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 888/1776] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:17:55 2022] DELETE /identity/v3/users/942e1e702362438384e98e0474353023 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:56.221763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d770a7c-cd4d-4568-a640-affa03c0adde None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:56.228426 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d770a7c-cd4d-4568-a640-affa03c0adde None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:56.228426 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d770a7c-cd4d-4568-a640-affa03c0adde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:56.292884 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4d770a7c-cd4d-4568-a640-affa03c0adde None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:56.296088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4d770a7c-cd4d-4568-a640-affa03c0adde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:56.391495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6cdc6-9868-446c-946f-795676ec58e6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:56.475472 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 889/1777] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:56.486209 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d770a7c-cd4d-4568-a640-affa03c0adde None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:56.487989 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4d770a7c-cd4d-4568-a640-affa03c0adde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:56.535954 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe9cba-4beb-4884-8949-2eb3cb70eb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:56.537946 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe9cba-4beb-4884-8949-2eb3cb70eb0a admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:56.538245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe9cba-4beb-4884-8949-2eb3cb70eb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:56.542347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe9cba-4beb-4884-8949-2eb3cb70eb0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:56.557974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe9cba-4beb-4884-8949-2eb3cb70eb0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:56.560752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe9cba-4beb-4884-8949-2eb3cb70eb0a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:56.616290 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d770a7c-cd4d-4568-a640-affa03c0adde None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:56.624727 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:56.641303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5223a8-6b70-4c8a-9799-5e62eadac8f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:56.645139 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5223a8-6b70-4c8a-9799-5e62eadac8f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:56.645968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5223a8-6b70-4c8a-9799-5e62eadac8f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:56.647102 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5223a8-6b70-4c8a-9799-5e62eadac8f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:56.670784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 889/1778] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:17:56 2022] POST /identity/v3/projects => generated 381 bytes in 461 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:56.686337 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe9cba-4beb-4884-8949-2eb3cb70eb0a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:56.690900 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0ffe9cba-4beb-4884-8949-2eb3cb70eb0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:56.723404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5223a8-6b70-4c8a-9799-5e62eadac8f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:56.733262 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5223a8-6b70-4c8a-9799-5e62eadac8f5 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:56.782316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 890/1779] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:56.930029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe9cba-4beb-4884-8949-2eb3cb70eb0a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:56.936181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fea94d-025b-4f61-8bb8-35a6a75ec082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:56.939670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fea94d-025b-4f61-8bb8-35a6a75ec082 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:56.939670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fea94d-025b-4f61-8bb8-35a6a75ec082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:56.939670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fea94d-025b-4f61-8bb8-35a6a75ec082 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:56.939670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fea94d-025b-4f61-8bb8-35a6a75ec082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:56.940633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fea94d-025b-4f61-8bb8-35a6a75ec082 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:57.053244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 890/1780] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:17:56 2022] POST /identity/v3/users => generated 297 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:17:57.057621 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5223a8-6b70-4c8a-9799-5e62eadac8f5 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:57.070833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5223a8-6b70-4c8a-9799-5e62eadac8f5 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:57.081871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e6016-23e4-4a0d-bb72-b6ba4cc7e14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:57.084648 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e6016-23e4-4a0d-bb72-b6ba4cc7e14d admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:57.085311 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e6016-23e4-4a0d-bb72-b6ba4cc7e14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:57.089977 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e6016-23e4-4a0d-bb72-b6ba4cc7e14d admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:57.089977 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e6016-23e4-4a0d-bb72-b6ba4cc7e14d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:57.089977 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e6016-23e4-4a0d-bb72-b6ba4cc7e14d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:57.115923 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 891/1781] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:57.120016 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fea94d-025b-4f61-8bb8-35a6a75ec082 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:57.135616 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fea94d-025b-4f61-8bb8-35a6a75ec082 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:57.153450 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103da1db-8657-45cb-9ea7-9ee474fd0983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:57.155051 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103da1db-8657-45cb-9ea7-9ee474fd0983 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:57.155862 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103da1db-8657-45cb-9ea7-9ee474fd0983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:57.156378 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103da1db-8657-45cb-9ea7-9ee474fd0983 admin admin] PATH_INFO: `/v3/projects/7c690b516198449e9db6e849d3ff0463/users/ab7e7e852c4e4301bde4882931ac2257/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:57.206019 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103da1db-8657-45cb-9ea7-9ee474fd0983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c690b516198449e9db6e849d3ff0463, user_id=ab7e7e852c4e4301bde4882931ac2257, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:57.240366 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103da1db-8657-45cb-9ea7-9ee474fd0983 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:57.348013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 891/1782] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:57 2022] PUT /identity/v3/projects/7c690b516198449e9db6e849d3ff0463/users/ab7e7e852c4e4301bde4882931ac2257/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:57.351943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e6016-23e4-4a0d-bb72-b6ba4cc7e14d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:57.375769 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e6016-23e4-4a0d-bb72-b6ba4cc7e14d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:57.523055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1572b-6700-486c-85a0-51d847853270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:57.524042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1572b-6700-486c-85a0-51d847853270 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:57.526272 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1572b-6700-486c-85a0-51d847853270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:57.529647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1572b-6700-486c-85a0-51d847853270 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:57.530627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c1572b-6700-486c-85a0-51d847853270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:57.532427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c1572b-6700-486c-85a0-51d847853270 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:57.554428 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103da1db-8657-45cb-9ea7-9ee474fd0983 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:57.562626 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 892/1783] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:17:57.579900 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103da1db-8657-45cb-9ea7-9ee474fd0983 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:57.595256 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38247930-1bf4-4929-9401-45e13cf9f5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:57.596248 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38247930-1bf4-4929-9401-45e13cf9f5e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:57.596624 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38247930-1bf4-4929-9401-45e13cf9f5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:57.596814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38247930-1bf4-4929-9401-45e13cf9f5e9 admin admin] PATH_INFO: `/v3/projects/7c690b516198449e9db6e849d3ff0463/users/ab7e7e852c4e4301bde4882931ac2257/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:57.599570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38247930-1bf4-4929-9401-45e13cf9f5e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c690b516198449e9db6e849d3ff0463, user_id=ab7e7e852c4e4301bde4882931ac2257, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:57.602245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38247930-1bf4-4929-9401-45e13cf9f5e9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:57.634202 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 892/1784] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:17:57 2022] PUT /identity/v3/projects/7c690b516198449e9db6e849d3ff0463/users/ab7e7e852c4e4301bde4882931ac2257/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:17:57.636434 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284568bb-fc07-4ae9-94fa-32228a007c33 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:57.641568 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284568bb-fc07-4ae9-94fa-32228a007c33 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:57.644858 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284568bb-fc07-4ae9-94fa-32228a007c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:57.765840 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-284568bb-fc07-4ae9-94fa-32228a007c33 None None] MFA Rules not processed for user `ab7e7e852c4e4301bde4882931ac2257`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:57.767298 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-284568bb-fc07-4ae9-94fa-32228a007c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:57.923563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 893/1785] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:17:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:57.940391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38247930-1bf4-4929-9401-45e13cf9f5e9 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:57.943364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-38247930-1bf4-4929-9401-45e13cf9f5e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:58.067128 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38247930-1bf4-4929-9401-45e13cf9f5e9 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:58.071419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172eed60-b9d6-4d66-9625-16596a09ab57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:58.072363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172eed60-b9d6-4d66-9625-16596a09ab57 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:58.072845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172eed60-b9d6-4d66-9625-16596a09ab57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:58.073482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172eed60-b9d6-4d66-9625-16596a09ab57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:58.096331 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172eed60-b9d6-4d66-9625-16596a09ab57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:58.098713 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172eed60-b9d6-4d66-9625-16596a09ab57 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:58.116431 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 893/1786] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:17:58.248537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14225773-e271-4aaa-be16-11b52abe7eb5 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:58.249649 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14225773-e271-4aaa-be16-11b52abe7eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:58.250316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14225773-e271-4aaa-be16-11b52abe7eb5 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:58.253474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 894/1787] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jun 24 04:17:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:17:58.275148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4f738-b424-4bb1-a021-e74a29aaec48 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:58.282139 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4f738-b424-4bb1-a021-e74a29aaec48 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:58.282139 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4f738-b424-4bb1-a021-e74a29aaec48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:58.378509 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-44c4f738-b424-4bb1-a021-e74a29aaec48 None None] MFA Rules not processed for user `80b7b3ec60db4de19bbf7d08f111f470`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:17:58.384623 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-44c4f738-b424-4bb1-a021-e74a29aaec48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:58.393657 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14225773-e271-4aaa-be16-11b52abe7eb5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:58.399506 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-14225773-e271-4aaa-be16-11b52abe7eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:58.432204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14225773-e271-4aaa-be16-11b52abe7eb5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:58.452116 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a04fab-4a18-40fb-9345-d130f569fb03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:58.455378 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a04fab-4a18-40fb-9345-d130f569fb03 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:58.460533 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a04fab-4a18-40fb-9345-d130f569fb03 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:58.460846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a04fab-4a18-40fb-9345-d130f569fb03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:58.469460 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:58.473692 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a9a04fab-4a18-40fb-9345-d130f569fb03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:58.542070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 894/1788] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jun 24 04:17:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:17:58.655740 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a04fab-4a18-40fb-9345-d130f569fb03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:58.662435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a04fab-4a18-40fb-9345-d130f569fb03 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:58.674297 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 895/1789] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:17:58.721403 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4f738-b424-4bb1-a021-e74a29aaec48 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:58.724963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-44c4f738-b424-4bb1-a021-e74a29aaec48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:58.875007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c4f738-b424-4bb1-a021-e74a29aaec48 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:58.893796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb1699-bc42-45c1-82d2-e0b7d86ffd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:58.912908 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb1699-bc42-45c1-82d2-e0b7d86ffd18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:58.912908 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb1699-bc42-45c1-82d2-e0b7d86ffd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:58.913809 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb1699-bc42-45c1-82d2-e0b7d86ffd18 admin admin] PATH_INFO: `/v3/projects/b7cce4ceff8f42b9bf990acd1127509d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:58.918839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb1699-bc42-45c1-82d2-e0b7d86ffd18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7cce4ceff8f42b9bf990acd1127509d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:58.925849 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fb1699-bc42-45c1-82d2-e0b7d86ffd18 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:59.038826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:17:59.130011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-23fb1699-bc42-45c1-82d2-e0b7d86ffd18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7cce4ceff8f42b9bf990acd1127509d', 'request_id': 'req-23fb1699-bc42-45c1-82d2-e0b7d86ffd18'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:59.137659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-23fb1699-bc42-45c1-82d2-e0b7d86ffd18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7cce4ceff8f42b9bf990acd1127509d', 'request_id': 'req-23fb1699-bc42-45c1-82d2-e0b7d86ffd18'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:59.145103 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-23fb1699-bc42-45c1-82d2-e0b7d86ffd18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7cce4ceff8f42b9bf990acd1127509d', 'request_id': 'req-23fb1699-bc42-45c1-82d2-e0b7d86ffd18'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:59.301558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-23fb1699-bc42-45c1-82d2-e0b7d86ffd18 admin admin] The token cache is being invalidate because project b7cce4ceff8f42b9bf990acd1127509d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:17:59.333353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-23fb1699-bc42-45c1-82d2-e0b7d86ffd18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:59.333353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-23fb1699-bc42-45c1-82d2-e0b7d86ffd18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:17:59.333353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 895/1790] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:17:58 2022] DELETE /identity/v3/projects/b7cce4ceff8f42b9bf990acd1127509d => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:17:59.333685 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a04fab-4a18-40fb-9345-d130f569fb03 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:59.361713 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a04fab-4a18-40fb-9345-d130f569fb03 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:59.449355 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efcc727-5d49-43e2-8c14-170b9ed8d658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:59.452487 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efcc727-5d49-43e2-8c14-170b9ed8d658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:59.453087 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efcc727-5d49-43e2-8c14-170b9ed8d658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:59.453518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efcc727-5d49-43e2-8c14-170b9ed8d658 admin admin] PATH_INFO: `/v3/users/0f7395f7dbdd4faf92dd98f963bedeb2` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:59.462563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efcc727-5d49-43e2-8c14-170b9ed8d658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f7395f7dbdd4faf92dd98f963bedeb2)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:59.465628 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efcc727-5d49-43e2-8c14-170b9ed8d658 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:59.567569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb1699-bc42-45c1-82d2-e0b7d86ffd18 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:17:59.569829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-23fb1699-bc42-45c1-82d2-e0b7d86ffd18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:59.742229 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb1699-bc42-45c1-82d2-e0b7d86ffd18 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:17:59.757597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb8344-be43-497c-b7ec-27a53f6babbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:17:59.758333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb8344-be43-497c-b7ec-27a53f6babbd service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:17:59.758675 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb8344-be43-497c-b7ec-27a53f6babbd service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:17:59.759120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb8344-be43-497c-b7ec-27a53f6babbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:17:59.763178 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-51bb8344-be43-497c-b7ec-27a53f6babbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:17:59.783917 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb8344-be43-497c-b7ec-27a53f6babbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:17:59.785892 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb8344-be43-497c-b7ec-27a53f6babbd service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:17:59.802145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 896/1791] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:17:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:00.035358 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9efcc727-5d49-43e2-8c14-170b9ed8d658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f7395f7dbdd4faf92dd98f963bedeb2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:00.059328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9efcc727-5d49-43e2-8c14-170b9ed8d658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f7395f7dbdd4faf92dd98f963bedeb2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:00.143292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9efcc727-5d49-43e2-8c14-170b9ed8d658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f7395f7dbdd4faf92dd98f963bedeb2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:00.143974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9efcc727-5d49-43e2-8c14-170b9ed8d658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f7395f7dbdd4faf92dd98f963bedeb2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:00.144807 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-9efcc727-5d49-43e2-8c14-170b9ed8d658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f7395f7dbdd4faf92dd98f963bedeb2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:00.170485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 896/1792] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:17:59 2022] DELETE /identity/v3/users/0f7395f7dbdd4faf92dd98f963bedeb2 => generated 0 bytes in 839 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:00.864804 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb8344-be43-497c-b7ec-27a53f6babbd service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:00.868098 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-51bb8344-be43-497c-b7ec-27a53f6babbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:00.908389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb8344-be43-497c-b7ec-27a53f6babbd service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:00.974215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885dcee7-691e-416e-98a6-75cdf15bc377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:00.975127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dcee7-691e-416e-98a6-75cdf15bc377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:00.975418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dcee7-691e-416e-98a6-75cdf15bc377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:00.975871 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885dcee7-691e-416e-98a6-75cdf15bc377 admin admin] PATH_INFO: `/v3/projects/448d79b9f1884604ac5d18edda555fd0` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:00.977153 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885dcee7-691e-416e-98a6-75cdf15bc377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=448d79b9f1884604ac5d18edda555fd0)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:00.979315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885dcee7-691e-416e-98a6-75cdf15bc377 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:01.094792 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-885dcee7-691e-416e-98a6-75cdf15bc377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '448d79b9f1884604ac5d18edda555fd0', 'request_id': 'req-885dcee7-691e-416e-98a6-75cdf15bc377'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:01.101504 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-885dcee7-691e-416e-98a6-75cdf15bc377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '448d79b9f1884604ac5d18edda555fd0', 'request_id': 'req-885dcee7-691e-416e-98a6-75cdf15bc377'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:01.107898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-885dcee7-691e-416e-98a6-75cdf15bc377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '448d79b9f1884604ac5d18edda555fd0', 'request_id': 'req-885dcee7-691e-416e-98a6-75cdf15bc377'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:01.183384 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-885dcee7-691e-416e-98a6-75cdf15bc377 admin admin] The token cache is being invalidate because project 448d79b9f1884604ac5d18edda555fd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:01.183675 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-885dcee7-691e-416e-98a6-75cdf15bc377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:01.184332 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-885dcee7-691e-416e-98a6-75cdf15bc377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:01.186225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 897/1793] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:00 2022] DELETE /identity/v3/projects/448d79b9f1884604ac5d18edda555fd0 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:01.194231 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873a4b7-2c42-4076-95d0-fa9f07df7055 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:01.194782 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873a4b7-2c42-4076-95d0-fa9f07df7055 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:01.194983 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873a4b7-2c42-4076-95d0-fa9f07df7055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:01.215880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0873a4b7-2c42-4076-95d0-fa9f07df7055 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:01.217847 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0873a4b7-2c42-4076-95d0-fa9f07df7055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:01.291218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 897/1794] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:01.298362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4046f52-0376-4b1e-96a4-7a5a548e7b43 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:01.299306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4046f52-0376-4b1e-96a4-7a5a548e7b43 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:01.299868 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4046f52-0376-4b1e-96a4-7a5a548e7b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:01.320226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b4046f52-0376-4b1e-96a4-7a5a548e7b43 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:01.321451 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b4046f52-0376-4b1e-96a4-7a5a548e7b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:01.334611 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 898/1795] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:01.335417 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873a4b7-2c42-4076-95d0-fa9f07df7055 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:01.352824 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873a4b7-2c42-4076-95d0-fa9f07df7055 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:01.359188 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7b382-706c-4c96-89b7-0c82b2a5f553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:01.359905 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b382-706c-4c96-89b7-0c82b2a5f553 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:01.360176 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b382-706c-4c96-89b7-0c82b2a5f553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:01.360637 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b382-706c-4c96-89b7-0c82b2a5f553 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:01.361507 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7b382-706c-4c96-89b7-0c82b2a5f553 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:18:01.361800 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7b382-706c-4c96-89b7-0c82b2a5f553 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:01.363468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7b382-706c-4c96-89b7-0c82b2a5f553 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:01.378458 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 898/1796] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:18:01.382125 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4046f52-0376-4b1e-96a4-7a5a548e7b43 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:01.391683 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4046f52-0376-4b1e-96a4-7a5a548e7b43 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:01.396645 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d73a8c-56a3-461b-a968-6c5cdceac0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:01.397311 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d73a8c-56a3-461b-a968-6c5cdceac0e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:01.397599 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d73a8c-56a3-461b-a968-6c5cdceac0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:01.397874 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d73a8c-56a3-461b-a968-6c5cdceac0e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:01.398646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d73a8c-56a3-461b-a968-6c5cdceac0e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:01.400567 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d73a8c-56a3-461b-a968-6c5cdceac0e2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:01.445936 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:01.529132 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 899/1797] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:18:01 2022] POST /identity/v3/projects => generated 363 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:01.533610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7b382-706c-4c96-89b7-0c82b2a5f553 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:01.543622 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7b382-706c-4c96-89b7-0c82b2a5f553 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:01.602882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad43a3a-ad97-4506-9820-974fdc3806b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:01.605268 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad43a3a-ad97-4506-9820-974fdc3806b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:01.605268 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad43a3a-ad97-4506-9820-974fdc3806b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:01.605268 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad43a3a-ad97-4506-9820-974fdc3806b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:01.605268 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad43a3a-ad97-4506-9820-974fdc3806b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:01.607112 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad43a3a-ad97-4506-9820-974fdc3806b9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:01.685141 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 899/1798] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:18:01 2022] POST /identity/v3/users => generated 282 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:01.693091 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d73a8c-56a3-461b-a968-6c5cdceac0e2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:01.718963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d73a8c-56a3-461b-a968-6c5cdceac0e2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:01.720092 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed2b2e-765b-41c9-be65-df10a107768a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:01.721089 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed2b2e-765b-41c9-be65-df10a107768a admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:01.721633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed2b2e-765b-41c9-be65-df10a107768a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:01.722120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed2b2e-765b-41c9-be65-df10a107768a admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:01.723074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed2b2e-765b-41c9-be65-df10a107768a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:01.725035 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed2b2e-765b-41c9-be65-df10a107768a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:01.743740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 900/1799] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:01.747416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad43a3a-ad97-4506-9820-974fdc3806b9 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:01.757264 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad43a3a-ad97-4506-9820-974fdc3806b9 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:01.768114 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e6cd7b-1f85-4cab-920c-06b646b5979e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:01.768927 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6cd7b-1f85-4cab-920c-06b646b5979e admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:01.769452 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6cd7b-1f85-4cab-920c-06b646b5979e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:01.769845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e6cd7b-1f85-4cab-920c-06b646b5979e admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:01.770775 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e6cd7b-1f85-4cab-920c-06b646b5979e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:01.776157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e6cd7b-1f85-4cab-920c-06b646b5979e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:01.790435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 900/1800] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:01.792986 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:01.794715 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed2b2e-765b-41c9-be65-df10a107768a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:01.805150 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed2b2e-765b-41c9-be65-df10a107768a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:01.810415 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e95f7e-c081-49d8-ac49-83e2f710f9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:01.810793 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e95f7e-c081-49d8-ac49-83e2f710f9c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:01.811127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e95f7e-c081-49d8-ac49-83e2f710f9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:01.811411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e95f7e-c081-49d8-ac49-83e2f710f9c1 admin admin] PATH_INFO: `/v3/projects/eef3e99567d140f69419fa851e149182/users/a5d80e9ff83147e49c82ace9f6780cff/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:01.827189 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e95f7e-c081-49d8-ac49-83e2f710f9c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eef3e99567d140f69419fa851e149182, user_id=a5d80e9ff83147e49c82ace9f6780cff, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:01.829898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e95f7e-c081-49d8-ac49-83e2f710f9c1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:01.926188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 901/1801] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:01 2022] PUT /identity/v3/projects/eef3e99567d140f69419fa851e149182/users/a5d80e9ff83147e49c82ace9f6780cff/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:01.938650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab61a7e-9e45-402b-ab7f-25615b7bf2e2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:01.939032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab61a7e-9e45-402b-ab7f-25615b7bf2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:01.939515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab61a7e-9e45-402b-ab7f-25615b7bf2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:01.981589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-aab61a7e-9e45-402b-ab7f-25615b7bf2e2 None None] MFA Rules not processed for user `a5d80e9ff83147e49c82ace9f6780cff`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:01.983030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-aab61a7e-9e45-402b-ab7f-25615b7bf2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:02.050623 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 901/1802] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:01 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:02.055509 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e95f7e-c081-49d8-ac49-83e2f710f9c1 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:02.123441 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e95f7e-c081-49d8-ac49-83e2f710f9c1 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:02.128663 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f87b8-839a-44e1-8753-4a105edb3a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:02.129392 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f87b8-839a-44e1-8753-4a105edb3a94 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:02.129691 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f87b8-839a-44e1-8753-4a105edb3a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:02.129968 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f87b8-839a-44e1-8753-4a105edb3a94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:02.130707 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f87b8-839a-44e1-8753-4a105edb3a94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:02.132716 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f87b8-839a-44e1-8753-4a105edb3a94 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:02.229399 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 902/1803] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:18:02 2022] POST /identity/v3/projects => generated 365 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:02.234714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab61a7e-9e45-402b-ab7f-25615b7bf2e2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:02.245268 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab61a7e-9e45-402b-ab7f-25615b7bf2e2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:02.301689 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9bcd2-5036-47f1-ace7-0f36be444786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:02.302294 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9bcd2-5036-47f1-ace7-0f36be444786 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:02.302612 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9bcd2-5036-47f1-ace7-0f36be444786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:02.302895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9bcd2-5036-47f1-ace7-0f36be444786 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:02.303610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9bcd2-5036-47f1-ace7-0f36be444786 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:02.305536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9bcd2-5036-47f1-ace7-0f36be444786 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:02.394413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 902/1804] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:18:02 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:02.395696 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f87b8-839a-44e1-8753-4a105edb3a94 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:02.406683 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f87b8-839a-44e1-8753-4a105edb3a94 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:02.411416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474878f4-e78a-410a-afef-fd0433fb07f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:02.412113 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474878f4-e78a-410a-afef-fd0433fb07f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:02.412402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474878f4-e78a-410a-afef-fd0433fb07f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:02.412685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474878f4-e78a-410a-afef-fd0433fb07f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:02.413426 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474878f4-e78a-410a-afef-fd0433fb07f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:02.415134 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474878f4-e78a-410a-afef-fd0433fb07f3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:02.426954 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 903/1805] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:02.433325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9bcd2-5036-47f1-ace7-0f36be444786 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:02.444568 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9bcd2-5036-47f1-ace7-0f36be444786 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:02.451947 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:02.465201 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9cfc4-fe92-402a-81cc-2782b6deea1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:02.466240 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9cfc4-fe92-402a-81cc-2782b6deea1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:02.466538 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9cfc4-fe92-402a-81cc-2782b6deea1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:02.466834 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9cfc4-fe92-402a-81cc-2782b6deea1d admin admin] PATH_INFO: `/v3/projects/f43a46013b904a83bd5ace4a88f31e3c/users/c1af4c8c450f4fc887e387978e746847/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:02.488915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9cfc4-fe92-402a-81cc-2782b6deea1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f43a46013b904a83bd5ace4a88f31e3c, user_id=c1af4c8c450f4fc887e387978e746847, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:02.491746 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9cfc4-fe92-402a-81cc-2782b6deea1d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:02.567447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 903/1806] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:02 2022] PUT /identity/v3/projects/f43a46013b904a83bd5ace4a88f31e3c/users/c1af4c8c450f4fc887e387978e746847/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:02.571415 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474878f4-e78a-410a-afef-fd0433fb07f3 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:02.582116 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474878f4-e78a-410a-afef-fd0433fb07f3 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:02.647426 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe4859-879f-46f5-829a-1d4814111c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:02.648401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe4859-879f-46f5-829a-1d4814111c12 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:02.648862 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe4859-879f-46f5-829a-1d4814111c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:02.649325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe4859-879f-46f5-829a-1d4814111c12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:02.650248 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe4859-879f-46f5-829a-1d4814111c12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:02.651911 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe4859-879f-46f5-829a-1d4814111c12 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:02.665250 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 904/1807] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:02.666668 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9cfc4-fe92-402a-81cc-2782b6deea1d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:02.677893 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9cfc4-fe92-402a-81cc-2782b6deea1d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:02.682772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3118583-3260-46e1-8ef8-53fad4823a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:02.683708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3118583-3260-46e1-8ef8-53fad4823a05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:02.683997 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3118583-3260-46e1-8ef8-53fad4823a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:02.690270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3118583-3260-46e1-8ef8-53fad4823a05 admin admin] PATH_INFO: `/v3/projects/f43a46013b904a83bd5ace4a88f31e3c/users/c1af4c8c450f4fc887e387978e746847/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:02.690270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3118583-3260-46e1-8ef8-53fad4823a05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f43a46013b904a83bd5ace4a88f31e3c, user_id=c1af4c8c450f4fc887e387978e746847, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:02.690270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3118583-3260-46e1-8ef8-53fad4823a05 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:02.701665 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 904/1808] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:02 2022] PUT /identity/v3/projects/f43a46013b904a83bd5ace4a88f31e3c/users/c1af4c8c450f4fc887e387978e746847/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:18:02.712178 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dbaa2-0aeb-415e-b460-91c54c6be987 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:02.713231 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dbaa2-0aeb-415e-b460-91c54c6be987 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:02.713495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620dbaa2-0aeb-415e-b460-91c54c6be987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:02.740176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-620dbaa2-0aeb-415e-b460-91c54c6be987 None None] MFA Rules not processed for user `c1af4c8c450f4fc887e387978e746847`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:02.741592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-620dbaa2-0aeb-415e-b460-91c54c6be987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:02.830535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 905/1809] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:02.846854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3118583-3260-46e1-8ef8-53fad4823a05 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:02.850467 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c3118583-3260-46e1-8ef8-53fad4823a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:02.916599 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3118583-3260-46e1-8ef8-53fad4823a05 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:02.921250 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64edba-6158-40d3-84d7-9ea54f5b4bed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:02.922309 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64edba-6158-40d3-84d7-9ea54f5b4bed service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:02.922680 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64edba-6158-40d3-84d7-9ea54f5b4bed service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:02.922962 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64edba-6158-40d3-84d7-9ea54f5b4bed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:02.932491 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64edba-6158-40d3-84d7-9ea54f5b4bed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:02.934032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64edba-6158-40d3-84d7-9ea54f5b4bed service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:02.944852 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 905/1810] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:03.184365 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dbaa2-0aeb-415e-b460-91c54c6be987 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:03.194294 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620dbaa2-0aeb-415e-b460-91c54c6be987 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:03.203454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5feddbc-ac9a-4eff-85f1-65b09467a41f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:03.203454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5feddbc-ac9a-4eff-85f1-65b09467a41f service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:03.203454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5feddbc-ac9a-4eff-85f1-65b09467a41f service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:03.203454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5feddbc-ac9a-4eff-85f1-65b09467a41f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:03.261851 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5feddbc-ac9a-4eff-85f1-65b09467a41f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:03.264121 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5feddbc-ac9a-4eff-85f1-65b09467a41f service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:03.273907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 906/1811] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:03 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:03.337396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:04.066259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:04.380863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:08.620067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3c640-68e7-4eda-8b0f-86003497c578 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:08.620597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3c640-68e7-4eda-8b0f-86003497c578 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:08.620674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3c640-68e7-4eda-8b0f-86003497c578 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:08.623291 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 906/1812] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jun 24 04:18:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:18:08.637061 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20edb2-ad6f-4b25-a9c2-992827c92ba9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:08.637061 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20edb2-ad6f-4b25-a9c2-992827c92ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:08.638428 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20edb2-ad6f-4b25-a9c2-992827c92ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:08.664981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4f20edb2-ad6f-4b25-a9c2-992827c92ba9 None None] MFA Rules not processed for user `80b7b3ec60db4de19bbf7d08f111f470`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:08.666807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4f20edb2-ad6f-4b25-a9c2-992827c92ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:08.758443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 907/1813] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jun 24 04:18:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:08.790542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3c640-68e7-4eda-8b0f-86003497c578 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:08.792621 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-62a3c640-68e7-4eda-8b0f-86003497c578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:08.812687 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3c640-68e7-4eda-8b0f-86003497c578 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:08.818508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7332253-3145-4c51-b19b-e001cc421912 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:08.819188 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7332253-3145-4c51-b19b-e001cc421912 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:08.819649 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7332253-3145-4c51-b19b-e001cc421912 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:08.819939 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7332253-3145-4c51-b19b-e001cc421912 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:08.830509 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7332253-3145-4c51-b19b-e001cc421912 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:08.832335 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7332253-3145-4c51-b19b-e001cc421912 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:08.866007 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 907/1814] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:09.216031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20edb2-ad6f-4b25-a9c2-992827c92ba9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:09.289587 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:09.326096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20edb2-ad6f-4b25-a9c2-992827c92ba9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:09.341608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd332da-60ad-4bd1-856f-55ff3a29ce2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:09.342818 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd332da-60ad-4bd1-856f-55ff3a29ce2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:09.343183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd332da-60ad-4bd1-856f-55ff3a29ce2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:09.343531 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd332da-60ad-4bd1-856f-55ff3a29ce2b admin admin] PATH_INFO: `/v3/users/a5d80e9ff83147e49c82ace9f6780cff` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:09.345043 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd332da-60ad-4bd1-856f-55ff3a29ce2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5d80e9ff83147e49c82ace9f6780cff)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:09.348318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd332da-60ad-4bd1-856f-55ff3a29ce2b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:09.625662 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1bd332da-60ad-4bd1-856f-55ff3a29ce2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5d80e9ff83147e49c82ace9f6780cff'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:09.649181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1bd332da-60ad-4bd1-856f-55ff3a29ce2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5d80e9ff83147e49c82ace9f6780cff'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:09.650663 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1bd332da-60ad-4bd1-856f-55ff3a29ce2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5d80e9ff83147e49c82ace9f6780cff'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:09.651796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1bd332da-60ad-4bd1-856f-55ff3a29ce2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5d80e9ff83147e49c82ace9f6780cff'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:09.663180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1bd332da-60ad-4bd1-856f-55ff3a29ce2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5d80e9ff83147e49c82ace9f6780cff'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:09.769903 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 908/1815] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:09 2022] DELETE /identity/v3/users/a5d80e9ff83147e49c82ace9f6780cff => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:09.774771 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286dd301-0667-455b-94c1-24168ea9c156 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:09.775234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286dd301-0667-455b-94c1-24168ea9c156 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:09.775408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286dd301-0667-455b-94c1-24168ea9c156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:09.795904 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-286dd301-0667-455b-94c1-24168ea9c156 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:09.795904 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-286dd301-0667-455b-94c1-24168ea9c156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:09.863895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 908/1816] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:09.874214 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd332da-60ad-4bd1-856f-55ff3a29ce2b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:09.875654 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1bd332da-60ad-4bd1-856f-55ff3a29ce2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:09.937274 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd332da-60ad-4bd1-856f-55ff3a29ce2b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:09.940927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b0806-1c77-431a-b509-10914bc17aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:09.941578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b0806-1c77-431a-b509-10914bc17aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:09.941872 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b0806-1c77-431a-b509-10914bc17aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:09.942149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b0806-1c77-431a-b509-10914bc17aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:09.951413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b0806-1c77-431a-b509-10914bc17aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:09.952883 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b0806-1c77-431a-b509-10914bc17aa0 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:09.962182 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 909/1817] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:10.890845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286dd301-0667-455b-94c1-24168ea9c156 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:10.903988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-286dd301-0667-455b-94c1-24168ea9c156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:10.946646 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286dd301-0667-455b-94c1-24168ea9c156 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:10.952499 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f2d70-dbd3-4d95-b3b9-ab2f3c549dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:10.953394 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f2d70-dbd3-4d95-b3b9-ab2f3c549dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:10.953729 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f2d70-dbd3-4d95-b3b9-ab2f3c549dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:10.955033 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f2d70-dbd3-4d95-b3b9-ab2f3c549dca admin admin] PATH_INFO: `/v3/projects/eef3e99567d140f69419fa851e149182` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:10.956369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f2d70-dbd3-4d95-b3b9-ab2f3c549dca admin admin] RBAC: Authorizing `identity:delete_project(project_id=eef3e99567d140f69419fa851e149182)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:10.971336 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3f2d70-dbd3-4d95-b3b9-ab2f3c549dca admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:11.130254 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b0806-1c77-431a-b509-10914bc17aa0 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:11.132827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1b7b0806-1c77-431a-b509-10914bc17aa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:11.148536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b0806-1c77-431a-b509-10914bc17aa0 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:11.152632 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bf3f2d70-dbd3-4d95-b3b9-ab2f3c549dca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eef3e99567d140f69419fa851e149182', 'request_id': 'req-bf3f2d70-dbd3-4d95-b3b9-ab2f3c549dca'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:11.205303 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f66fd-0e60-4cb8-bcfd-c53f3df19b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:11.206180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f66fd-0e60-4cb8-bcfd-c53f3df19b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:11.206504 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f66fd-0e60-4cb8-bcfd-c53f3df19b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:11.206790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f66fd-0e60-4cb8-bcfd-c53f3df19b39 admin admin] PATH_INFO: `/v3/users/a2b315c8801943118874ea6240bb8d1c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:11.207980 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f66fd-0e60-4cb8-bcfd-c53f3df19b39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2b315c8801943118874ea6240bb8d1c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:11.210967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bf3f2d70-dbd3-4d95-b3b9-ab2f3c549dca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eef3e99567d140f69419fa851e149182', 'request_id': 'req-bf3f2d70-dbd3-4d95-b3b9-ab2f3c549dca'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:11.211618 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37f66fd-0e60-4cb8-bcfd-c53f3df19b39 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:11.214017 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:11.218011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bf3f2d70-dbd3-4d95-b3b9-ab2f3c549dca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eef3e99567d140f69419fa851e149182', 'request_id': 'req-bf3f2d70-dbd3-4d95-b3b9-ab2f3c549dca'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:11.230930 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bf3f2d70-dbd3-4d95-b3b9-ab2f3c549dca admin admin] The token cache is being invalidate because project eef3e99567d140f69419fa851e149182 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:11.232109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bf3f2d70-dbd3-4d95-b3b9-ab2f3c549dca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:11.233030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bf3f2d70-dbd3-4d95-b3b9-ab2f3c549dca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:11.234743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 909/1818] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:10 2022] DELETE /identity/v3/projects/eef3e99567d140f69419fa851e149182 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:11.239649 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f2d70-dbd3-4d95-b3b9-ab2f3c549dca admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:11.241089 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bf3f2d70-dbd3-4d95-b3b9-ab2f3c549dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:11.254902 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f2d70-dbd3-4d95-b3b9-ab2f3c549dca admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:11.259855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143121fc-42c8-463d-859a-c81ac4c04b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:11.260974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143121fc-42c8-463d-859a-c81ac4c04b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:11.261269 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143121fc-42c8-463d-859a-c81ac4c04b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:11.261762 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143121fc-42c8-463d-859a-c81ac4c04b82 admin admin] PATH_INFO: `/v3/users/c1af4c8c450f4fc887e387978e746847` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:11.263675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143121fc-42c8-463d-859a-c81ac4c04b82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1af4c8c450f4fc887e387978e746847)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:11.265953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143121fc-42c8-463d-859a-c81ac4c04b82 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:11.384896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:11.422749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d37f66fd-0e60-4cb8-bcfd-c53f3df19b39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2b315c8801943118874ea6240bb8d1c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:11.438708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d37f66fd-0e60-4cb8-bcfd-c53f3df19b39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2b315c8801943118874ea6240bb8d1c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:11.439453 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d37f66fd-0e60-4cb8-bcfd-c53f3df19b39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2b315c8801943118874ea6240bb8d1c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:11.440072 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d37f66fd-0e60-4cb8-bcfd-c53f3df19b39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2b315c8801943118874ea6240bb8d1c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:11.448035 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d37f66fd-0e60-4cb8-bcfd-c53f3df19b39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2b315c8801943118874ea6240bb8d1c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:11.473673 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-143121fc-42c8-463d-859a-c81ac4c04b82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1af4c8c450f4fc887e387978e746847'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:11.486931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-143121fc-42c8-463d-859a-c81ac4c04b82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1af4c8c450f4fc887e387978e746847'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:11.561479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 910/1819] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:11 2022] DELETE /identity/v3/users/a2b315c8801943118874ea6240bb8d1c => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:11.563000 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b208e-83e8-4163-8470-a9f70456c5c9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:11.563293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b208e-83e8-4163-8470-a9f70456c5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:11.563572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b208e-83e8-4163-8470-a9f70456c5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:11.581565 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b43b208e-83e8-4163-8470-a9f70456c5c9 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:11.582792 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b43b208e-83e8-4163-8470-a9f70456c5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:11.642077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 911/1820] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:11.645305 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-143121fc-42c8-463d-859a-c81ac4c04b82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1af4c8c450f4fc887e387978e746847'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:11.645961 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-143121fc-42c8-463d-859a-c81ac4c04b82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1af4c8c450f4fc887e387978e746847'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:11.646854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-143121fc-42c8-463d-859a-c81ac4c04b82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1af4c8c450f4fc887e387978e746847'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:11.652084 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b208e-83e8-4163-8470-a9f70456c5c9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:11.653552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b43b208e-83e8-4163-8470-a9f70456c5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:11.660882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 910/1821] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:11 2022] DELETE /identity/v3/users/c1af4c8c450f4fc887e387978e746847 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:18:11.712564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b208e-83e8-4163-8470-a9f70456c5c9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:11.751346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:11.772444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29698a66-211d-4fac-b907-36f7c781c35d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:11.773113 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29698a66-211d-4fac-b907-36f7c781c35d service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:11.773417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29698a66-211d-4fac-b907-36f7c781c35d service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:11.773688 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29698a66-211d-4fac-b907-36f7c781c35d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:11.775620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-29698a66-211d-4fac-b907-36f7c781c35d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:11.851110 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29698a66-211d-4fac-b907-36f7c781c35d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:11.852821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29698a66-211d-4fac-b907-36f7c781c35d service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:11.865053 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 912/1822] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:18:12.438886 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143121fc-42c8-463d-859a-c81ac4c04b82 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:12.440833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-143121fc-42c8-463d-859a-c81ac4c04b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:12.463627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143121fc-42c8-463d-859a-c81ac4c04b82 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:12.469538 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d66ba2-1c54-4770-a51f-2f5bb33ed6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:12.470683 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d66ba2-1c54-4770-a51f-2f5bb33ed6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:12.471228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d66ba2-1c54-4770-a51f-2f5bb33ed6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:12.471565 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d66ba2-1c54-4770-a51f-2f5bb33ed6ee admin admin] PATH_INFO: `/v3/projects/f43a46013b904a83bd5ace4a88f31e3c` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:12.473189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d66ba2-1c54-4770-a51f-2f5bb33ed6ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=f43a46013b904a83bd5ace4a88f31e3c)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:12.475428 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d66ba2-1c54-4770-a51f-2f5bb33ed6ee admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:12.644133 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-12d66ba2-1c54-4770-a51f-2f5bb33ed6ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f43a46013b904a83bd5ace4a88f31e3c', 'request_id': 'req-12d66ba2-1c54-4770-a51f-2f5bb33ed6ee'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:12.713186 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29698a66-211d-4fac-b907-36f7c781c35d service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:12.714729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-29698a66-211d-4fac-b907-36f7c781c35d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:12.744525 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29698a66-211d-4fac-b907-36f7c781c35d service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:12.759980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-12d66ba2-1c54-4770-a51f-2f5bb33ed6ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f43a46013b904a83bd5ace4a88f31e3c', 'request_id': 'req-12d66ba2-1c54-4770-a51f-2f5bb33ed6ee'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:12.773976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-12d66ba2-1c54-4770-a51f-2f5bb33ed6ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f43a46013b904a83bd5ace4a88f31e3c', 'request_id': 'req-12d66ba2-1c54-4770-a51f-2f5bb33ed6ee'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:12.788571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-12d66ba2-1c54-4770-a51f-2f5bb33ed6ee admin admin] The token cache is being invalidate because project f43a46013b904a83bd5ace4a88f31e3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:12.788932 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-12d66ba2-1c54-4770-a51f-2f5bb33ed6ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:12.789528 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-12d66ba2-1c54-4770-a51f-2f5bb33ed6ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:12.791123 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 911/1823] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:12 2022] DELETE /identity/v3/projects/f43a46013b904a83bd5ace4a88f31e3c => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:12.797027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5112a-dcc8-4f2e-8bc2-c4715a5e0776 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:12.797389 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5112a-dcc8-4f2e-8bc2-c4715a5e0776 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:12.803579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5112a-dcc8-4f2e-8bc2-c4715a5e0776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:12.817557 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-50b5112a-dcc8-4f2e-8bc2-c4715a5e0776 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:12.818938 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-50b5112a-dcc8-4f2e-8bc2-c4715a5e0776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:12.829288 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8f302-8243-4d27-8e44-78d321407bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:12.832482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8f302-8243-4d27-8e44-78d321407bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:12.832482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8f302-8243-4d27-8e44-78d321407bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:12.832482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8f302-8243-4d27-8e44-78d321407bdd admin admin] PATH_INFO: `/v3/projects/cff4c8b1bc24457181baf5a594b1b8fc` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:12.832482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8f302-8243-4d27-8e44-78d321407bdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cff4c8b1bc24457181baf5a594b1b8fc)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:12.834396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d8f302-8243-4d27-8e44-78d321407bdd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:12.889980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 912/1824] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:18:12.898197 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fbbb5-8ab1-4877-a67d-476d0afacbf1 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:12.898555 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fbbb5-8ab1-4877-a67d-476d0afacbf1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:12.898835 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fbbb5-8ab1-4877-a67d-476d0afacbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:12.918001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a26fbbb5-8ab1-4877-a67d-476d0afacbf1 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:12.919328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a26fbbb5-8ab1-4877-a67d-476d0afacbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:12.927306 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 913/1825] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:18:12.931278 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26fbbb5-8ab1-4877-a67d-476d0afacbf1 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:12.940324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26fbbb5-8ab1-4877-a67d-476d0afacbf1 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:12.944926 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425f5cf-af13-4c0a-b3b7-5327a15a2177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:12.945582 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425f5cf-af13-4c0a-b3b7-5327a15a2177 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:12.945847 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425f5cf-af13-4c0a-b3b7-5327a15a2177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:12.946121 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425f5cf-af13-4c0a-b3b7-5327a15a2177 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:12.946876 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425f5cf-af13-4c0a-b3b7-5327a15a2177 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:18:12.947181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425f5cf-af13-4c0a-b3b7-5327a15a2177 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:12.948765 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425f5cf-af13-4c0a-b3b7-5327a15a2177 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:12.961200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 914/1826] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:18:12.965243 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425f5cf-af13-4c0a-b3b7-5327a15a2177 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:12.976582 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425f5cf-af13-4c0a-b3b7-5327a15a2177 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:12.984048 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:12.989469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-89d8f302-8243-4d27-8e44-78d321407bdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cff4c8b1bc24457181baf5a594b1b8fc', 'request_id': 'req-89d8f302-8243-4d27-8e44-78d321407bdd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:12.999328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-89d8f302-8243-4d27-8e44-78d321407bdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cff4c8b1bc24457181baf5a594b1b8fc', 'request_id': 'req-89d8f302-8243-4d27-8e44-78d321407bdd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:13.007304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-89d8f302-8243-4d27-8e44-78d321407bdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cff4c8b1bc24457181baf5a594b1b8fc', 'request_id': 'req-89d8f302-8243-4d27-8e44-78d321407bdd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:13.053993 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb7fec-45ce-43da-b589-628cfec37772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:13.054867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb7fec-45ce-43da-b589-628cfec37772 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:13.055221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb7fec-45ce-43da-b589-628cfec37772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:13.055606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb7fec-45ce-43da-b589-628cfec37772 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:13.056390 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb7fec-45ce-43da-b589-628cfec37772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:13.058480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb7fec-45ce-43da-b589-628cfec37772 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:13.175176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-89d8f302-8243-4d27-8e44-78d321407bdd admin admin] The token cache is being invalidate because project cff4c8b1bc24457181baf5a594b1b8fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:13.175470 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-89d8f302-8243-4d27-8e44-78d321407bdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:13.176104 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-89d8f302-8243-4d27-8e44-78d321407bdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:13.177723 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 913/1827] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:12 2022] DELETE /identity/v3/projects/cff4c8b1bc24457181baf5a594b1b8fc => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:13.181389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8f302-8243-4d27-8e44-78d321407bdd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:13.182961 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-89d8f302-8243-4d27-8e44-78d321407bdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:13.195706 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8f302-8243-4d27-8e44-78d321407bdd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:13.206757 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 915/1828] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:18:12 2022] POST /identity/v3/projects => generated 377 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:18:13.211046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb7fec-45ce-43da-b589-628cfec37772 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:13.213093 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-35fb7fec-45ce-43da-b589-628cfec37772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:13.213917 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45887fe-a638-4a9c-bb5a-1e7e8e3b1c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:13.214992 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45887fe-a638-4a9c-bb5a-1e7e8e3b1c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:13.215344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45887fe-a638-4a9c-bb5a-1e7e8e3b1c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:13.216273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45887fe-a638-4a9c-bb5a-1e7e8e3b1c64 admin admin] PATH_INFO: `/v3/users/ab7e7e852c4e4301bde4882931ac2257` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:13.217545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45887fe-a638-4a9c-bb5a-1e7e8e3b1c64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab7e7e852c4e4301bde4882931ac2257)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:13.219677 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45887fe-a638-4a9c-bb5a-1e7e8e3b1c64 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:13.237057 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:13.282598 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb7fec-45ce-43da-b589-628cfec37772 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:13.294285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f677756-f901-4400-a64c-9dbb993e2faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:13.295014 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f677756-f901-4400-a64c-9dbb993e2faf admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:13.295303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f677756-f901-4400-a64c-9dbb993e2faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:13.295581 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f677756-f901-4400-a64c-9dbb993e2faf admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:13.296294 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f677756-f901-4400-a64c-9dbb993e2faf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:13.298231 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f677756-f901-4400-a64c-9dbb993e2faf admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:13.411237 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 916/1829] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:18:13 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:13.415513 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f677756-f901-4400-a64c-9dbb993e2faf admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:13.437093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f45887fe-a638-4a9c-bb5a-1e7e8e3b1c64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab7e7e852c4e4301bde4882931ac2257'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:13.437421 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f677756-f901-4400-a64c-9dbb993e2faf admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:13.442791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32b139-a5a3-475d-b8c1-1acbb31b7187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:13.444239 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32b139-a5a3-475d-b8c1-1acbb31b7187 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:13.444672 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32b139-a5a3-475d-b8c1-1acbb31b7187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:13.445239 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32b139-a5a3-475d-b8c1-1acbb31b7187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:13.446280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e32b139-a5a3-475d-b8c1-1acbb31b7187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:13.460722 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f45887fe-a638-4a9c-bb5a-1e7e8e3b1c64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab7e7e852c4e4301bde4882931ac2257'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:13.466355 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e32b139-a5a3-475d-b8c1-1acbb31b7187 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:13.468113 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f45887fe-a638-4a9c-bb5a-1e7e8e3b1c64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab7e7e852c4e4301bde4882931ac2257'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:13.468810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f45887fe-a638-4a9c-bb5a-1e7e8e3b1c64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab7e7e852c4e4301bde4882931ac2257'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:13.482824 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 917/1830] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:13.484585 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32b139-a5a3-475d-b8c1-1acbb31b7187 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:13.487558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f45887fe-a638-4a9c-bb5a-1e7e8e3b1c64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab7e7e852c4e4301bde4882931ac2257'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:13.488173 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8e32b139-a5a3-475d-b8c1-1acbb31b7187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:13.508211 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32b139-a5a3-475d-b8c1-1acbb31b7187 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:13.514112 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e151a-e8bd-4b63-8bc2-6e3b26d1c8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:13.515442 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e151a-e8bd-4b63-8bc2-6e3b26d1c8cc admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:13.515442 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e151a-e8bd-4b63-8bc2-6e3b26d1c8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:13.515728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e151a-e8bd-4b63-8bc2-6e3b26d1c8cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:13.516476 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e151a-e8bd-4b63-8bc2-6e3b26d1c8cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:13.518122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e151a-e8bd-4b63-8bc2-6e3b26d1c8cc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:13.530197 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 918/1831] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:13.534598 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e151a-e8bd-4b63-8bc2-6e3b26d1c8cc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:13.546105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e151a-e8bd-4b63-8bc2-6e3b26d1c8cc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:13.556376 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c07116-e110-4610-9dfd-c11282c3484f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:13.557282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c07116-e110-4610-9dfd-c11282c3484f admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:13.557570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c07116-e110-4610-9dfd-c11282c3484f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:13.557867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c07116-e110-4610-9dfd-c11282c3484f admin admin] PATH_INFO: `/v3/projects/9597f37257e640b095fae407357fa2b9/users/bc62214a1f4043209d2c6d0c88c2b058/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:13.570619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 914/1832] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:13 2022] DELETE /identity/v3/users/ab7e7e852c4e4301bde4882931ac2257 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:18:13.577687 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c07116-e110-4610-9dfd-c11282c3484f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9597f37257e640b095fae407357fa2b9, user_id=bc62214a1f4043209d2c6d0c88c2b058, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:13.580757 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c07116-e110-4610-9dfd-c11282c3484f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:13.718237 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 919/1833] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:13 2022] PUT /identity/v3/projects/9597f37257e640b095fae407357fa2b9/users/bc62214a1f4043209d2c6d0c88c2b058/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:13.729259 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245f556-4553-4043-8839-464a72376a99 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:13.729749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245f556-4553-4043-8839-464a72376a99 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:13.729842 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4245f556-4553-4043-8839-464a72376a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:13.756495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4245f556-4553-4043-8839-464a72376a99 None None] MFA Rules not processed for user `bc62214a1f4043209d2c6d0c88c2b058`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:13.757701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4245f556-4553-4043-8839-464a72376a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:13.852314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 915/1834] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:13.857886 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c07116-e110-4610-9dfd-c11282c3484f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:13.913104 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:13.937974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c07116-e110-4610-9dfd-c11282c3484f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:13.943292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557147d7-1a8e-467e-9ab3-1816d43dbd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:13.943989 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557147d7-1a8e-467e-9ab3-1816d43dbd07 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:13.944349 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557147d7-1a8e-467e-9ab3-1816d43dbd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:13.944631 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557147d7-1a8e-467e-9ab3-1816d43dbd07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:13.945499 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557147d7-1a8e-467e-9ab3-1816d43dbd07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:13.947601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557147d7-1a8e-467e-9ab3-1816d43dbd07 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:14.029107 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 920/1835] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:18:13 2022] POST /identity/v3/projects => generated 379 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:14.033004 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245f556-4553-4043-8839-464a72376a99 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:14.051506 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4245f556-4553-4043-8839-464a72376a99 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:14.123348 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf474c5-1f2f-4759-96fe-987f9e9549ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:14.124034 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf474c5-1f2f-4759-96fe-987f9e9549ab admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:14.124320 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf474c5-1f2f-4759-96fe-987f9e9549ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:14.124602 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf474c5-1f2f-4759-96fe-987f9e9549ab admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:14.125302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf474c5-1f2f-4759-96fe-987f9e9549ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:14.127459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf474c5-1f2f-4759-96fe-987f9e9549ab admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:14.219508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557147d7-1a8e-467e-9ab3-1816d43dbd07 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:14.221437 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-557147d7-1a8e-467e-9ab3-1816d43dbd07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:14.240239 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557147d7-1a8e-467e-9ab3-1816d43dbd07 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:14.244535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 916/1836] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:18:14 2022] POST /identity/v3/users => generated 290 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:14.248586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf474c5-1f2f-4759-96fe-987f9e9549ab admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:14.253832 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee71fda-bd4c-4292-9748-f8f76676dca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:14.254743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee71fda-bd4c-4292-9748-f8f76676dca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:14.255029 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee71fda-bd4c-4292-9748-f8f76676dca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:14.255308 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee71fda-bd4c-4292-9748-f8f76676dca0 admin admin] PATH_INFO: `/v3/projects/7c690b516198449e9db6e849d3ff0463` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:14.256755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee71fda-bd4c-4292-9748-f8f76676dca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c690b516198449e9db6e849d3ff0463)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:14.258977 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee71fda-bd4c-4292-9748-f8f76676dca0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:14.263886 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf474c5-1f2f-4759-96fe-987f9e9549ab admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:14.269867 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ffde56-7cbf-47ee-b5de-436ef4bbfab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:14.270881 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffde56-7cbf-47ee-b5de-436ef4bbfab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:14.271182 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffde56-7cbf-47ee-b5de-436ef4bbfab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:14.271735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffde56-7cbf-47ee-b5de-436ef4bbfab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:14.272510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ffde56-7cbf-47ee-b5de-436ef4bbfab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:14.274349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ffde56-7cbf-47ee-b5de-436ef4bbfab9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:14.281666 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:14.290093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 917/1837] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:14.291142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ffde56-7cbf-47ee-b5de-436ef4bbfab9 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:14.301440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ffde56-7cbf-47ee-b5de-436ef4bbfab9 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:14.306907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34365eb0-d1ef-4932-960e-2589e6cd5414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:14.307726 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34365eb0-d1ef-4932-960e-2589e6cd5414 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:14.308124 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34365eb0-d1ef-4932-960e-2589e6cd5414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:14.308511 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34365eb0-d1ef-4932-960e-2589e6cd5414 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:14.309432 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34365eb0-d1ef-4932-960e-2589e6cd5414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:14.311325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34365eb0-d1ef-4932-960e-2589e6cd5414 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:14.333843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 918/1838] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:14.338084 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34365eb0-d1ef-4932-960e-2589e6cd5414 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:14.348419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34365eb0-d1ef-4932-960e-2589e6cd5414 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:14.353140 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57b687-f7a0-4f69-9d7c-377d1faceb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:14.356809 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57b687-f7a0-4f69-9d7c-377d1faceb73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:14.356809 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57b687-f7a0-4f69-9d7c-377d1faceb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:14.356809 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57b687-f7a0-4f69-9d7c-377d1faceb73 admin admin] PATH_INFO: `/v3/projects/9dfea382aea84994924e5110ed6757ec/users/93dcc4168895488fb19d9038b5741420/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:14.370305 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57b687-f7a0-4f69-9d7c-377d1faceb73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dfea382aea84994924e5110ed6757ec, user_id=93dcc4168895488fb19d9038b5741420, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:14.373905 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57b687-f7a0-4f69-9d7c-377d1faceb73 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:14.475605 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 919/1839] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:14 2022] PUT /identity/v3/projects/9dfea382aea84994924e5110ed6757ec/users/93dcc4168895488fb19d9038b5741420/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:14.487387 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624d1f2-8863-46f7-ad21-0c686970d65f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:14.487746 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624d1f2-8863-46f7-ad21-0c686970d65f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:14.489279 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624d1f2-8863-46f7-ad21-0c686970d65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:14.489379 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7ee71fda-bd4c-4292-9748-f8f76676dca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c690b516198449e9db6e849d3ff0463', 'request_id': 'req-7ee71fda-bd4c-4292-9748-f8f76676dca0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:14.524862 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9624d1f2-8863-46f7-ad21-0c686970d65f None None] MFA Rules not processed for user `93dcc4168895488fb19d9038b5741420`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:14.526096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9624d1f2-8863-46f7-ad21-0c686970d65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:14.593755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7ee71fda-bd4c-4292-9748-f8f76676dca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c690b516198449e9db6e849d3ff0463', 'request_id': 'req-7ee71fda-bd4c-4292-9748-f8f76676dca0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:14.601858 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7ee71fda-bd4c-4292-9748-f8f76676dca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c690b516198449e9db6e849d3ff0463', 'request_id': 'req-7ee71fda-bd4c-4292-9748-f8f76676dca0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:14.615258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 920/1840] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:18:14.619340 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624d1f2-8863-46f7-ad21-0c686970d65f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:14.628949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7ee71fda-bd4c-4292-9748-f8f76676dca0 admin admin] The token cache is being invalidate because project 7c690b516198449e9db6e849d3ff0463 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:14.629651 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7ee71fda-bd4c-4292-9748-f8f76676dca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:14.631083 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-7ee71fda-bd4c-4292-9748-f8f76676dca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:14.635901 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 921/1841] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:14 2022] DELETE /identity/v3/projects/7c690b516198449e9db6e849d3ff0463 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:14.641550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae647ec1-6498-400b-a70c-e3138cba2bd2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:14.642032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae647ec1-6498-400b-a70c-e3138cba2bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:14.642495 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae647ec1-6498-400b-a70c-e3138cba2bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:14.666630 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ae647ec1-6498-400b-a70c-e3138cba2bd2 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:14.667878 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ae647ec1-6498-400b-a70c-e3138cba2bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:14.755043 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624d1f2-8863-46f7-ad21-0c686970d65f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:14.765616 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0299ba7-ebb5-4120-9005-ed375532b4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:14.766306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0299ba7-ebb5-4120-9005-ed375532b4c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:14.766632 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0299ba7-ebb5-4120-9005-ed375532b4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:14.776883 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0299ba7-ebb5-4120-9005-ed375532b4c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:14.776883 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0299ba7-ebb5-4120-9005-ed375532b4c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:14.776883 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0299ba7-ebb5-4120-9005-ed375532b4c7 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:14.792820 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 922/1842] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:18:14.794157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7e8f9-b84d-48ac-8b26-3445575546a2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:14.794711 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7e8f9-b84d-48ac-8b26-3445575546a2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:14.794792 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7e8f9-b84d-48ac-8b26-3445575546a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:14.855625 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-71a7e8f9-b84d-48ac-8b26-3445575546a2 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:14.858126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 921/1843] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:18:14 2022] POST /identity/v3/projects => generated 379 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:14.859394 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-71a7e8f9-b84d-48ac-8b26-3445575546a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:14.861889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0299ba7-ebb5-4120-9005-ed375532b4c7 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:14.863538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c0299ba7-ebb5-4120-9005-ed375532b4c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:14.930243 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:14.933672 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:14.955911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 923/1844] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:14.961136 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7e8f9-b84d-48ac-8b26-3445575546a2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:14.968500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0299ba7-ebb5-4120-9005-ed375532b4c7 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:14.978805 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95596aed-c39c-4ada-baa4-2563f2038800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:14.979625 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95596aed-c39c-4ada-baa4-2563f2038800 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:14.980029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95596aed-c39c-4ada-baa4-2563f2038800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:14.980378 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95596aed-c39c-4ada-baa4-2563f2038800 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:14.981877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7e8f9-b84d-48ac-8b26-3445575546a2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:14.982573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95596aed-c39c-4ada-baa4-2563f2038800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:14.984870 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95596aed-c39c-4ada-baa4-2563f2038800 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:14.986956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd66f1-7c1f-4450-aa80-66ef7e0c50c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:14.991027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd66f1-7c1f-4450-aa80-66ef7e0c50c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:14.991027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd66f1-7c1f-4450-aa80-66ef7e0c50c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:14.991027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd66f1-7c1f-4450-aa80-66ef7e0c50c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:14.991027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd66f1-7c1f-4450-aa80-66ef7e0c50c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:18:14.991027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd66f1-7c1f-4450-aa80-66ef7e0c50c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:14.991027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd66f1-7c1f-4450-aa80-66ef7e0c50c1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:15.005170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 924/1845] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:18:15.009396 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd66f1-7c1f-4450-aa80-66ef7e0c50c1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:15.019713 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd66f1-7c1f-4450-aa80-66ef7e0c50c1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:15.024699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd398863-8827-4337-8e77-68a72c3267f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:15.025347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd398863-8827-4337-8e77-68a72c3267f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:15.025635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd398863-8827-4337-8e77-68a72c3267f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:15.025915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd398863-8827-4337-8e77-68a72c3267f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:15.026645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd398863-8827-4337-8e77-68a72c3267f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:15.028631 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd398863-8827-4337-8e77-68a72c3267f2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:15.076301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 922/1846] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:18:14 2022] POST /identity/v3/users => generated 296 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:18:15.077368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95596aed-c39c-4ada-baa4-2563f2038800 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:15.087461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95596aed-c39c-4ada-baa4-2563f2038800 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:15.092712 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f62b74-e53c-4542-a23c-73777bba4d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:15.097780 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f62b74-e53c-4542-a23c-73777bba4d46 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:15.097780 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f62b74-e53c-4542-a23c-73777bba4d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:15.097780 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f62b74-e53c-4542-a23c-73777bba4d46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:15.097780 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f62b74-e53c-4542-a23c-73777bba4d46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:15.097780 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f62b74-e53c-4542-a23c-73777bba4d46 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:15.107262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 923/1847] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:18:15.111402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f62b74-e53c-4542-a23c-73777bba4d46 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:15.120959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f62b74-e53c-4542-a23c-73777bba4d46 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:15.126966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e7cd6-e671-497b-9fa7-ed461313d9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:15.127966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e7cd6-e671-497b-9fa7-ed461313d9eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:15.128260 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e7cd6-e671-497b-9fa7-ed461313d9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:15.128567 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e7cd6-e671-497b-9fa7-ed461313d9eb admin admin] PATH_INFO: `/v3/projects/e110cf156df446e2b2c342181f177651/users/bd883844c8a74d29ad92d1fe45bb870e/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:15.143915 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e7cd6-e671-497b-9fa7-ed461313d9eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e110cf156df446e2b2c342181f177651, user_id=bd883844c8a74d29ad92d1fe45bb870e, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:15.146913 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032e7cd6-e671-497b-9fa7-ed461313d9eb admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:15.165520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 925/1848] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:18:15 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:18:15.170848 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd398863-8827-4337-8e77-68a72c3267f2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:15.238603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 924/1849] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:15 2022] PUT /identity/v3/projects/e110cf156df446e2b2c342181f177651/users/bd883844c8a74d29ad92d1fe45bb870e/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:18:15.240345 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd398863-8827-4337-8e77-68a72c3267f2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:15.243459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e7cd6-e671-497b-9fa7-ed461313d9eb admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:15.310977 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61913522-b8a8-4008-9cde-2cb635be545a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:15.311662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61913522-b8a8-4008-9cde-2cb635be545a admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:15.311942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61913522-b8a8-4008-9cde-2cb635be545a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:15.312213 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61913522-b8a8-4008-9cde-2cb635be545a admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:15.312911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61913522-b8a8-4008-9cde-2cb635be545a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:15.313918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032e7cd6-e671-497b-9fa7-ed461313d9eb admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:15.314888 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61913522-b8a8-4008-9cde-2cb635be545a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:15.318780 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71326c7c-3200-4837-b87d-b9f67ce2c421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:15.319538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71326c7c-3200-4837-b87d-b9f67ce2c421 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:15.320256 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71326c7c-3200-4837-b87d-b9f67ce2c421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:15.320256 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71326c7c-3200-4837-b87d-b9f67ce2c421 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:15.321058 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71326c7c-3200-4837-b87d-b9f67ce2c421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:15.322937 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71326c7c-3200-4837-b87d-b9f67ce2c421 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:15.339835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 925/1850] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:15.344393 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71326c7c-3200-4837-b87d-b9f67ce2c421 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:15.356031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71326c7c-3200-4837-b87d-b9f67ce2c421 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:15.361649 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafbe200-4303-4dc4-805c-3b8aeb1d19e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:15.362659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbe200-4303-4dc4-805c-3b8aeb1d19e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:15.363039 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbe200-4303-4dc4-805c-3b8aeb1d19e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:15.363386 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafbe200-4303-4dc4-805c-3b8aeb1d19e6 admin admin] PATH_INFO: `/v3/projects/e110cf156df446e2b2c342181f177651/users/bd883844c8a74d29ad92d1fe45bb870e/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:15.365816 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafbe200-4303-4dc4-805c-3b8aeb1d19e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e110cf156df446e2b2c342181f177651, user_id=bd883844c8a74d29ad92d1fe45bb870e, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:15.368623 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafbe200-4303-4dc4-805c-3b8aeb1d19e6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:15.382837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 926/1851] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:15 2022] PUT /identity/v3/projects/e110cf156df446e2b2c342181f177651/users/bd883844c8a74d29ad92d1fe45bb870e/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:15.393168 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9246fac-cb57-48f3-8de4-16d4312853b7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:15.393545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9246fac-cb57-48f3-8de4-16d4312853b7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:15.393931 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9246fac-cb57-48f3-8de4-16d4312853b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:15.426107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e9246fac-cb57-48f3-8de4-16d4312853b7 None None] MFA Rules not processed for user `bd883844c8a74d29ad92d1fe45bb870e`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:15.427372 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e9246fac-cb57-48f3-8de4-16d4312853b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:15.442868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 926/1852] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:18:15 2022] POST /identity/v3/users => generated 287 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:18:15.444165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61913522-b8a8-4008-9cde-2cb635be545a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:15.456274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61913522-b8a8-4008-9cde-2cb635be545a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:15.507149 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:15.508709 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:15.514007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 927/1853] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:15.525935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9246fac-cb57-48f3-8de4-16d4312853b7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:15.528661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e9246fac-cb57-48f3-8de4-16d4312853b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:15.531557 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d0c5d-fd37-4e68-82b1-f332a568d4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:15.532334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d0c5d-fd37-4e68-82b1-f332a568d4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:15.533055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d0c5d-fd37-4e68-82b1-f332a568d4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:15.533055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d0c5d-fd37-4e68-82b1-f332a568d4b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:15.533609 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d0c5d-fd37-4e68-82b1-f332a568d4b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:15.535286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d0c5d-fd37-4e68-82b1-f332a568d4b0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:15.550232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 927/1854] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:15.551032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d0c5d-fd37-4e68-82b1-f332a568d4b0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:15.561451 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d0c5d-fd37-4e68-82b1-f332a568d4b0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:15.566634 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1deb8e4-8292-4cb9-93d6-32891648a449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:15.567259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1deb8e4-8292-4cb9-93d6-32891648a449 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:15.567552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1deb8e4-8292-4cb9-93d6-32891648a449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:15.567829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1deb8e4-8292-4cb9-93d6-32891648a449 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:15.568561 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1deb8e4-8292-4cb9-93d6-32891648a449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:15.570163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1deb8e4-8292-4cb9-93d6-32891648a449 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:15.587650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 928/1855] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:15.597770 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1deb8e4-8292-4cb9-93d6-32891648a449 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:15.602822 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9246fac-cb57-48f3-8de4-16d4312853b7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:15.607896 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7598c00-6987-48e7-b00d-e053cad211c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:15.608630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7598c00-6987-48e7-b00d-e053cad211c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:15.608729 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1deb8e4-8292-4cb9-93d6-32891648a449 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:15.609019 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7598c00-6987-48e7-b00d-e053cad211c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:15.609763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7598c00-6987-48e7-b00d-e053cad211c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:15.613625 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6f7cb-7922-4ff6-bddf-be03edbf2ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:15.614557 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6f7cb-7922-4ff6-bddf-be03edbf2ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:15.614846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6f7cb-7922-4ff6-bddf-be03edbf2ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:15.615359 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6f7cb-7922-4ff6-bddf-be03edbf2ec2 admin admin] PATH_INFO: `/v3/projects/007af11c2fc7458ba9ec2325bc5dbf12/users/ba0297b65c0b4d3bb3ffabc00a862751/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:15.619948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7598c00-6987-48e7-b00d-e053cad211c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:15.621759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7598c00-6987-48e7-b00d-e053cad211c2 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:15.632812 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d6f7cb-7922-4ff6-bddf-be03edbf2ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=007af11c2fc7458ba9ec2325bc5dbf12, user_id=ba0297b65c0b4d3bb3ffabc00a862751, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:15.633588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 928/1856] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:18:15.635677 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d6f7cb-7922-4ff6-bddf-be03edbf2ec2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:15.720253 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 929/1857] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:15 2022] PUT /identity/v3/projects/007af11c2fc7458ba9ec2325bc5dbf12/users/ba0297b65c0b4d3bb3ffabc00a862751/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:15.725977 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f5fe5-b954-46f5-99a0-33d5605d3bb0 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:15.726268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f5fe5-b954-46f5-99a0-33d5605d3bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:15.726583 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f5fe5-b954-46f5-99a0-33d5605d3bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:15.760732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6f7cb-7922-4ff6-bddf-be03edbf2ec2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:15.764108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f4d6f7cb-7922-4ff6-bddf-be03edbf2ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:15.766445 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-1f5f5fe5-b954-46f5-99a0-33d5605d3bb0 None None] MFA Rules not processed for user `ba0297b65c0b4d3bb3ffabc00a862751`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:15.767791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1f5f5fe5-b954-46f5-99a0-33d5605d3bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:15.866028 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6f7cb-7922-4ff6-bddf-be03edbf2ec2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:15.871079 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 929/1858] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:15.877688 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f5fe5-b954-46f5-99a0-33d5605d3bb0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:15.880340 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95791b-79f3-437a-88eb-18d4c4396aae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:15.883614 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95791b-79f3-437a-88eb-18d4c4396aae service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:15.884001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95791b-79f3-437a-88eb-18d4c4396aae service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:15.884294 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95791b-79f3-437a-88eb-18d4c4396aae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:15.891962 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6a95791b-79f3-437a-88eb-18d4c4396aae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:15.971775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f5fe5-b954-46f5-99a0-33d5605d3bb0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:15.979411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c87aa-0430-49b6-8c79-9a15a1d4f4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:15.980356 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c87aa-0430-49b6-8c79-9a15a1d4f4b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:15.980524 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c87aa-0430-49b6-8c79-9a15a1d4f4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:15.980909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c87aa-0430-49b6-8c79-9a15a1d4f4b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:15.982342 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c87aa-0430-49b6-8c79-9a15a1d4f4b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:15.984523 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c87aa-0430-49b6-8c79-9a15a1d4f4b6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:15.998091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95791b-79f3-437a-88eb-18d4c4396aae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:15.999788 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a95791b-79f3-437a-88eb-18d4c4396aae service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:16.010468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 930/1859] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:16.115726 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 930/1860] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:18:15 2022] POST /identity/v3/projects => generated 373 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:16.117213 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95791b-79f3-437a-88eb-18d4c4396aae service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:16.135353 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95791b-79f3-437a-88eb-18d4c4396aae service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:16.234854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a251005-d6d3-4784-8eb2-c12e241a5544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:16.235664 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a251005-d6d3-4784-8eb2-c12e241a5544 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:16.236010 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a251005-d6d3-4784-8eb2-c12e241a5544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:16.236353 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a251005-d6d3-4784-8eb2-c12e241a5544 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:16.237128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a251005-d6d3-4784-8eb2-c12e241a5544 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:16.239124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a251005-d6d3-4784-8eb2-c12e241a5544 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:16.248709 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:16.354405 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 931/1861] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:18:16 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:18:16.358243 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c87aa-0430-49b6-8c79-9a15a1d4f4b6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:16.369148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c87aa-0430-49b6-8c79-9a15a1d4f4b6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:16.374330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ec82a-9dcf-4b2d-b180-9ebf42f44d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:16.375333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ec82a-9dcf-4b2d-b180-9ebf42f44d7b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:16.375680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ec82a-9dcf-4b2d-b180-9ebf42f44d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:16.375966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ec82a-9dcf-4b2d-b180-9ebf42f44d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:16.376697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ec82a-9dcf-4b2d-b180-9ebf42f44d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:16.378355 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ec82a-9dcf-4b2d-b180-9ebf42f44d7b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:16.390531 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 931/1862] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:16.394790 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a251005-d6d3-4784-8eb2-c12e241a5544 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:16.405502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a251005-d6d3-4784-8eb2-c12e241a5544 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:16.410976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08e3d4-8f52-4444-a798-2c29671a5ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:16.414325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08e3d4-8f52-4444-a798-2c29671a5ad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:16.414325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08e3d4-8f52-4444-a798-2c29671a5ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:16.414325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e08e3d4-8f52-4444-a798-2c29671a5ad1 admin admin] PATH_INFO: `/v3/projects/fe446fead0f64738b6d339ebb3d1c74e/users/0040091cee94487cb66b4ba319da4c52/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:16.428113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08e3d4-8f52-4444-a798-2c29671a5ad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe446fead0f64738b6d339ebb3d1c74e, user_id=0040091cee94487cb66b4ba319da4c52, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:16.430823 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e08e3d4-8f52-4444-a798-2c29671a5ad1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:16.532077 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 932/1863] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:16 2022] PUT /identity/v3/projects/fe446fead0f64738b6d339ebb3d1c74e/users/0040091cee94487cb66b4ba319da4c52/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:16.536027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ec82a-9dcf-4b2d-b180-9ebf42f44d7b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:16.546533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ec82a-9dcf-4b2d-b180-9ebf42f44d7b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:16.550276 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:16.618066 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb33d8-ddb8-416f-8034-0ec3a9fa25db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:16.618904 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb33d8-ddb8-416f-8034-0ec3a9fa25db admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:16.619261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb33d8-ddb8-416f-8034-0ec3a9fa25db admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:16.619469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb33d8-ddb8-416f-8034-0ec3a9fa25db admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:16.620327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb33d8-ddb8-416f-8034-0ec3a9fa25db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:16.622361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb33d8-ddb8-416f-8034-0ec3a9fa25db admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:16.634176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 932/1864] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:16 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:16.639199 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08e3d4-8f52-4444-a798-2c29671a5ad1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:16.651089 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e08e3d4-8f52-4444-a798-2c29671a5ad1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:16.657764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f2ee9-a8de-4767-97e6-589389862edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:16.659007 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f2ee9-a8de-4767-97e6-589389862edf admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:16.659295 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f2ee9-a8de-4767-97e6-589389862edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:16.659574 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f2ee9-a8de-4767-97e6-589389862edf admin admin] PATH_INFO: `/v3/projects/fe446fead0f64738b6d339ebb3d1c74e/users/0040091cee94487cb66b4ba319da4c52/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:16.661811 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f2ee9-a8de-4767-97e6-589389862edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe446fead0f64738b6d339ebb3d1c74e, user_id=0040091cee94487cb66b4ba319da4c52, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:16.664699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f2ee9-a8de-4767-97e6-589389862edf admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:16.679294 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 933/1865] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:16 2022] PUT /identity/v3/projects/fe446fead0f64738b6d339ebb3d1c74e/users/0040091cee94487cb66b4ba319da4c52/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:18:16.690652 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ab752-db3d-4fbb-8da3-5cb19a3e49d8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:16.690944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ab752-db3d-4fbb-8da3-5cb19a3e49d8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:16.691223 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ab752-db3d-4fbb-8da3-5cb19a3e49d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:16.723226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f57ab752-db3d-4fbb-8da3-5cb19a3e49d8 None None] MFA Rules not processed for user `0040091cee94487cb66b4ba319da4c52`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:16.724706 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f57ab752-db3d-4fbb-8da3-5cb19a3e49d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:16.800000 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 933/1866] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:16 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:16.811612 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f2ee9-a8de-4767-97e6-589389862edf admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:16.878170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f2ee9-a8de-4767-97e6-589389862edf admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:16.882543 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd620c8-c0c3-4ca0-8d91-76351e7a5ec5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:16.883208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd620c8-c0c3-4ca0-8d91-76351e7a5ec5 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:16.883496 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd620c8-c0c3-4ca0-8d91-76351e7a5ec5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:16.883803 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd620c8-c0c3-4ca0-8d91-76351e7a5ec5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:16.898282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd620c8-c0c3-4ca0-8d91-76351e7a5ec5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:16.900372 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd620c8-c0c3-4ca0-8d91-76351e7a5ec5 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:16.912058 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 934/1867] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:17.967061 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ab752-db3d-4fbb-8da3-5cb19a3e49d8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:18.035841 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ab752-db3d-4fbb-8da3-5cb19a3e49d8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:18.040618 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae85905-b042-4c84-99f9-0860da4f3b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:18.041605 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae85905-b042-4c84-99f9-0860da4f3b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:18.041964 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae85905-b042-4c84-99f9-0860da4f3b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:18.042319 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae85905-b042-4c84-99f9-0860da4f3b35 admin admin] PATH_INFO: `/v3/users/ba0297b65c0b4d3bb3ffabc00a862751` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:18.043691 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae85905-b042-4c84-99f9-0860da4f3b35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba0297b65c0b4d3bb3ffabc00a862751)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:18.045707 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae85905-b042-4c84-99f9-0860da4f3b35 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:18.204193 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:18.217589 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0ae85905-b042-4c84-99f9-0860da4f3b35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba0297b65c0b4d3bb3ffabc00a862751'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:18.229542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0ae85905-b042-4c84-99f9-0860da4f3b35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba0297b65c0b4d3bb3ffabc00a862751'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:18.230434 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0ae85905-b042-4c84-99f9-0860da4f3b35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba0297b65c0b4d3bb3ffabc00a862751'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:18.231648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0ae85905-b042-4c84-99f9-0860da4f3b35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba0297b65c0b4d3bb3ffabc00a862751'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:18.238491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0ae85905-b042-4c84-99f9-0860da4f3b35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba0297b65c0b4d3bb3ffabc00a862751'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:18.312314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 934/1868] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:17 2022] DELETE /identity/v3/users/ba0297b65c0b4d3bb3ffabc00a862751 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:18.317044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a5a04-5cc2-4047-82c0-f75eabc42afc None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:18.317490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a5a04-5cc2-4047-82c0-f75eabc42afc None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:18.317610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a5a04-5cc2-4047-82c0-f75eabc42afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:18.335749 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d85a5a04-5cc2-4047-82c0-f75eabc42afc None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:18.336997 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d85a5a04-5cc2-4047-82c0-f75eabc42afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:18.396399 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 935/1869] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:18.406686 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae85905-b042-4c84-99f9-0860da4f3b35 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:18.408117 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0ae85905-b042-4c84-99f9-0860da4f3b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:18.477280 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae85905-b042-4c84-99f9-0860da4f3b35 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:18.482319 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83d8cc-3933-4848-af80-d8c90eb7a080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:18.483018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83d8cc-3933-4848-af80-d8c90eb7a080 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:18.483298 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83d8cc-3933-4848-af80-d8c90eb7a080 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:18.483571 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83d8cc-3933-4848-af80-d8c90eb7a080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:18.493841 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83d8cc-3933-4848-af80-d8c90eb7a080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:18.495428 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83d8cc-3933-4848-af80-d8c90eb7a080 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:18.506578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 935/1870] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:18.975788 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a5a04-5cc2-4047-82c0-f75eabc42afc None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:18.977226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d85a5a04-5cc2-4047-82c0-f75eabc42afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:18.992416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85a5a04-5cc2-4047-82c0-f75eabc42afc None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:19.004805 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999eac53-9f59-49bd-b93a-3a4bee149d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:19.006001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999eac53-9f59-49bd-b93a-3a4bee149d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:19.006257 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999eac53-9f59-49bd-b93a-3a4bee149d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:19.006699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999eac53-9f59-49bd-b93a-3a4bee149d51 admin admin] PATH_INFO: `/v3/projects/007af11c2fc7458ba9ec2325bc5dbf12` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:19.008214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999eac53-9f59-49bd-b93a-3a4bee149d51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=007af11c2fc7458ba9ec2325bc5dbf12)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:19.011021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999eac53-9f59-49bd-b93a-3a4bee149d51 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:19.183862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-999eac53-9f59-49bd-b93a-3a4bee149d51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '007af11c2fc7458ba9ec2325bc5dbf12', 'request_id': 'req-999eac53-9f59-49bd-b93a-3a4bee149d51'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:19.188248 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:INSERT=4,keystone:SELECT=91,keystone:DELETE=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:19.308510 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-999eac53-9f59-49bd-b93a-3a4bee149d51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '007af11c2fc7458ba9ec2325bc5dbf12', 'request_id': 'req-999eac53-9f59-49bd-b93a-3a4bee149d51'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:19.325734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-999eac53-9f59-49bd-b93a-3a4bee149d51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '007af11c2fc7458ba9ec2325bc5dbf12', 'request_id': 'req-999eac53-9f59-49bd-b93a-3a4bee149d51'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:19.339662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-999eac53-9f59-49bd-b93a-3a4bee149d51 admin admin] The token cache is being invalidate because project 007af11c2fc7458ba9ec2325bc5dbf12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:19.340607 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-999eac53-9f59-49bd-b93a-3a4bee149d51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:19.341615 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-999eac53-9f59-49bd-b93a-3a4bee149d51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:19.343860 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 936/1871] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:18 2022] DELETE /identity/v3/projects/007af11c2fc7458ba9ec2325bc5dbf12 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:19.349585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83d8cc-3933-4848-af80-d8c90eb7a080 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:19.364577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83d8cc-3933-4848-af80-d8c90eb7a080 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:19.423129 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde444e-1bea-4895-855b-93edd2c7c2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:19.424014 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde444e-1bea-4895-855b-93edd2c7c2f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:19.424299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde444e-1bea-4895-855b-93edd2c7c2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:19.424631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde444e-1bea-4895-855b-93edd2c7c2f3 admin admin] PATH_INFO: `/v3/users/0040091cee94487cb66b4ba319da4c52` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:19.425826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde444e-1bea-4895-855b-93edd2c7c2f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0040091cee94487cb66b4ba319da4c52)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:19.432946 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cde444e-1bea-4895-855b-93edd2c7c2f3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:19.670570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7cde444e-1bea-4895-855b-93edd2c7c2f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0040091cee94487cb66b4ba319da4c52'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:19.678870 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:19.690637 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7cde444e-1bea-4895-855b-93edd2c7c2f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0040091cee94487cb66b4ba319da4c52'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:19.691687 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7cde444e-1bea-4895-855b-93edd2c7c2f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0040091cee94487cb66b4ba319da4c52'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:19.692371 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7cde444e-1bea-4895-855b-93edd2c7c2f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0040091cee94487cb66b4ba319da4c52'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:19.702213 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7cde444e-1bea-4895-855b-93edd2c7c2f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0040091cee94487cb66b4ba319da4c52'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:19.788465 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 936/1872] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:19 2022] DELETE /identity/v3/users/0040091cee94487cb66b4ba319da4c52 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:18:20.388012 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999eac53-9f59-49bd-b93a-3a4bee149d51 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:20.390170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-999eac53-9f59-49bd-b93a-3a4bee149d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:20.407927 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999eac53-9f59-49bd-b93a-3a4bee149d51 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:20.467827 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0373a7a-b478-4bdb-84e5-0e352d45d586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:20.468683 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0373a7a-b478-4bdb-84e5-0e352d45d586 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:20.468962 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0373a7a-b478-4bdb-84e5-0e352d45d586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:20.469233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0373a7a-b478-4bdb-84e5-0e352d45d586 admin admin] PATH_INFO: `/v3/projects/fe446fead0f64738b6d339ebb3d1c74e` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:20.470799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0373a7a-b478-4bdb-84e5-0e352d45d586 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe446fead0f64738b6d339ebb3d1c74e)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:20.472928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0373a7a-b478-4bdb-84e5-0e352d45d586 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:20.597641 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a0373a7a-b478-4bdb-84e5-0e352d45d586 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe446fead0f64738b6d339ebb3d1c74e', 'request_id': 'req-a0373a7a-b478-4bdb-84e5-0e352d45d586'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:20.679608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a0373a7a-b478-4bdb-84e5-0e352d45d586 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe446fead0f64738b6d339ebb3d1c74e', 'request_id': 'req-a0373a7a-b478-4bdb-84e5-0e352d45d586'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:20.687135 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a0373a7a-b478-4bdb-84e5-0e352d45d586 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe446fead0f64738b6d339ebb3d1c74e', 'request_id': 'req-a0373a7a-b478-4bdb-84e5-0e352d45d586'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:20.699957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a0373a7a-b478-4bdb-84e5-0e352d45d586 admin admin] The token cache is being invalidate because project fe446fead0f64738b6d339ebb3d1c74e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:20.701019 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a0373a7a-b478-4bdb-84e5-0e352d45d586 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:20.702071 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a0373a7a-b478-4bdb-84e5-0e352d45d586 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:20.704739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 937/1873] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:20 2022] DELETE /identity/v3/projects/fe446fead0f64738b6d339ebb3d1c74e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:20.711814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113665e4-369d-473f-8ba0-cd9b65a65757 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:20.712279 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113665e4-369d-473f-8ba0-cd9b65a65757 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:20.712501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113665e4-369d-473f-8ba0-cd9b65a65757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:20.733751 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-113665e4-369d-473f-8ba0-cd9b65a65757 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:20.735051 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-113665e4-369d-473f-8ba0-cd9b65a65757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:20.797705 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 937/1874] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:20.806148 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dfb33-95b3-4495-bec1-805462b59c2d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:20.806634 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dfb33-95b3-4495-bec1-805462b59c2d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:20.806980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dfb33-95b3-4495-bec1-805462b59c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:20.831384 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-493dfb33-95b3-4495-bec1-805462b59c2d None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:20.833662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-493dfb33-95b3-4495-bec1-805462b59c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:20.852125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 938/1875] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:20.853095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113665e4-369d-473f-8ba0-cd9b65a65757 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:20.863839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113665e4-369d-473f-8ba0-cd9b65a65757 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:20.868862 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04b1e4-28b3-4cad-8021-4a9f083277de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:20.869600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04b1e4-28b3-4cad-8021-4a9f083277de admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:20.869951 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04b1e4-28b3-4cad-8021-4a9f083277de admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:20.870296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04b1e4-28b3-4cad-8021-4a9f083277de admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:20.871136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04b1e4-28b3-4cad-8021-4a9f083277de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:18:20.871671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04b1e4-28b3-4cad-8021-4a9f083277de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:20.873349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04b1e4-28b3-4cad-8021-4a9f083277de admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:20.889092 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 938/1876] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:18:20.893426 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493dfb33-95b3-4495-bec1-805462b59c2d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:20.903670 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493dfb33-95b3-4495-bec1-805462b59c2d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:20.910502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e147f2-5ec0-4002-9a8a-3f454fa7e02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:20.911218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e147f2-5ec0-4002-9a8a-3f454fa7e02b admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:20.914256 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e147f2-5ec0-4002-9a8a-3f454fa7e02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:20.914256 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e147f2-5ec0-4002-9a8a-3f454fa7e02b admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:20.914256 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e147f2-5ec0-4002-9a8a-3f454fa7e02b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:20.914661 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e147f2-5ec0-4002-9a8a-3f454fa7e02b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:21.062410 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 939/1877] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:18:20 2022] POST /identity/v3/projects => generated 361 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:21.065797 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04b1e4-28b3-4cad-8021-4a9f083277de admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:21.076879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04b1e4-28b3-4cad-8021-4a9f083277de admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:21.134853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edacb1a-f11c-4c28-b8db-bca41611cbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:21.135589 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edacb1a-f11c-4c28-b8db-bca41611cbaf admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:21.135897 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edacb1a-f11c-4c28-b8db-bca41611cbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:21.136177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edacb1a-f11c-4c28-b8db-bca41611cbaf admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:21.140182 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edacb1a-f11c-4c28-b8db-bca41611cbaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:21.140182 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edacb1a-f11c-4c28-b8db-bca41611cbaf admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:21.245461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 939/1878] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:18:21 2022] POST /identity/v3/users => generated 281 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:18:21.250929 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e147f2-5ec0-4002-9a8a-3f454fa7e02b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:21.261372 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e147f2-5ec0-4002-9a8a-3f454fa7e02b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:21.266320 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe6823-405b-42cb-a619-16398ca6a3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:21.267012 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe6823-405b-42cb-a619-16398ca6a3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:21.267300 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe6823-405b-42cb-a619-16398ca6a3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:21.267575 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe6823-405b-42cb-a619-16398ca6a3b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:21.268301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbe6823-405b-42cb-a619-16398ca6a3b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:21.269893 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbe6823-405b-42cb-a619-16398ca6a3b7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:21.283142 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 940/1879] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:21.287147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edacb1a-f11c-4c28-b8db-bca41611cbaf admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:21.298728 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edacb1a-f11c-4c28-b8db-bca41611cbaf admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:21.305799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5c36b-3b35-4c07-8e75-7c31c5e5049d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:21.307019 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5c36b-3b35-4c07-8e75-7c31c5e5049d admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:21.307531 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5c36b-3b35-4c07-8e75-7c31c5e5049d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:21.312128 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5c36b-3b35-4c07-8e75-7c31c5e5049d admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:21.312128 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5c36b-3b35-4c07-8e75-7c31c5e5049d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:21.312128 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5c36b-3b35-4c07-8e75-7c31c5e5049d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:21.321470 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 940/1880] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:21.326413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe6823-405b-42cb-a619-16398ca6a3b7 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:21.337617 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe6823-405b-42cb-a619-16398ca6a3b7 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:21.342903 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d1725-c8a4-4225-b870-04c4f11f013b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:21.343799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d1725-c8a4-4225-b870-04c4f11f013b admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:21.344085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d1725-c8a4-4225-b870-04c4f11f013b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:21.344363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d1725-c8a4-4225-b870-04c4f11f013b admin admin] PATH_INFO: `/v3/projects/97caa6530ed54751bcda16beeb647672/users/29779f869b9a49cb8e750112a89e37f5/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:21.360574 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d1725-c8a4-4225-b870-04c4f11f013b admin admin] RBAC: Authorizing `identity:create_grant(project_id=97caa6530ed54751bcda16beeb647672, user_id=29779f869b9a49cb8e750112a89e37f5, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:21.365258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d1725-c8a4-4225-b870-04c4f11f013b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:21.487711 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 941/1881] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:21 2022] PUT /identity/v3/projects/97caa6530ed54751bcda16beeb647672/users/29779f869b9a49cb8e750112a89e37f5/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:21.499239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8adb9-49c0-4baf-bb1b-6674be5be054 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:21.500213 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8adb9-49c0-4baf-bb1b-6674be5be054 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:21.501040 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8adb9-49c0-4baf-bb1b-6674be5be054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:21.533760 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8bd8adb9-49c0-4baf-bb1b-6674be5be054 None None] MFA Rules not processed for user `29779f869b9a49cb8e750112a89e37f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:21.535110 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8bd8adb9-49c0-4baf-bb1b-6674be5be054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:21.560888 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:21.604082 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 941/1882] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:21.638991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d1725-c8a4-4225-b870-04c4f11f013b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:21.641442 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bb4d1725-c8a4-4225-b870-04c4f11f013b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:21.654494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:21.702224 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d1725-c8a4-4225-b870-04c4f11f013b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:21.706000 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70c3e3-d62e-4b38-bf22-963119928631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:21.706706 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70c3e3-d62e-4b38-bf22-963119928631 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:21.706996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70c3e3-d62e-4b38-bf22-963119928631 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:21.707304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70c3e3-d62e-4b38-bf22-963119928631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:21.716949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70c3e3-d62e-4b38-bf22-963119928631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:21.718641 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70c3e3-d62e-4b38-bf22-963119928631 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:21.728020 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 942/1883] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:21 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:21.743050 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8adb9-49c0-4baf-bb1b-6674be5be054 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:21.823796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8adb9-49c0-4baf-bb1b-6674be5be054 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:21.829260 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe01950-ae84-4c36-877d-bb5c8f0aeb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:21.830230 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe01950-ae84-4c36-877d-bb5c8f0aeb59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:21.830621 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe01950-ae84-4c36-877d-bb5c8f0aeb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:21.830935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe01950-ae84-4c36-877d-bb5c8f0aeb59 admin admin] PATH_INFO: `/v3/users/29779f869b9a49cb8e750112a89e37f5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:21.832179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe01950-ae84-4c36-877d-bb5c8f0aeb59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29779f869b9a49cb8e750112a89e37f5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:21.834131 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe01950-ae84-4c36-877d-bb5c8f0aeb59 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:22.133143 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5fe01950-ae84-4c36-877d-bb5c8f0aeb59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29779f869b9a49cb8e750112a89e37f5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:22.155195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5fe01950-ae84-4c36-877d-bb5c8f0aeb59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29779f869b9a49cb8e750112a89e37f5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:22.155989 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5fe01950-ae84-4c36-877d-bb5c8f0aeb59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29779f869b9a49cb8e750112a89e37f5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:22.156686 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5fe01950-ae84-4c36-877d-bb5c8f0aeb59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29779f869b9a49cb8e750112a89e37f5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:22.163579 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5fe01950-ae84-4c36-877d-bb5c8f0aeb59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29779f869b9a49cb8e750112a89e37f5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:22.233928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 942/1884] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:21 2022] DELETE /identity/v3/users/29779f869b9a49cb8e750112a89e37f5 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:22.238769 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088173a-94e2-405d-9d2d-7fddefca781f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:22.239063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088173a-94e2-405d-9d2d-7fddefca781f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:22.239338 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088173a-94e2-405d-9d2d-7fddefca781f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:22.260263 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6088173a-94e2-405d-9d2d-7fddefca781f None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:22.261958 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6088173a-94e2-405d-9d2d-7fddefca781f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:22.330059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 943/1885] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:22.339799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe01950-ae84-4c36-877d-bb5c8f0aeb59 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:22.342135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5fe01950-ae84-4c36-877d-bb5c8f0aeb59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:22.399692 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe01950-ae84-4c36-877d-bb5c8f0aeb59 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:22.403977 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e7483-d4dd-470d-8020-2aa07b06644f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:22.404630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e7483-d4dd-470d-8020-2aa07b06644f service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:22.404917 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e7483-d4dd-470d-8020-2aa07b06644f service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:22.405200 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e7483-d4dd-470d-8020-2aa07b06644f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:22.414349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e7483-d4dd-470d-8020-2aa07b06644f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:22.415897 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e7483-d4dd-470d-8020-2aa07b06644f service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:22.426073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 943/1886] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:22.599104 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088173a-94e2-405d-9d2d-7fddefca781f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:22.601374 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6088173a-94e2-405d-9d2d-7fddefca781f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:22.616471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088173a-94e2-405d-9d2d-7fddefca781f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:22.623601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e541cb-0a65-4426-a44a-b6f978b2d9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:22.624848 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e541cb-0a65-4426-a44a-b6f978b2d9d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:22.625337 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e541cb-0a65-4426-a44a-b6f978b2d9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:22.625809 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e541cb-0a65-4426-a44a-b6f978b2d9d8 admin admin] PATH_INFO: `/v3/users/bc62214a1f4043209d2c6d0c88c2b058` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:22.628672 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e541cb-0a65-4426-a44a-b6f978b2d9d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc62214a1f4043209d2c6d0c88c2b058)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:22.630670 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e541cb-0a65-4426-a44a-b6f978b2d9d8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:22.674210 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:22.847059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-55e541cb-0a65-4426-a44a-b6f978b2d9d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc62214a1f4043209d2c6d0c88c2b058'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:22.864610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-55e541cb-0a65-4426-a44a-b6f978b2d9d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc62214a1f4043209d2c6d0c88c2b058'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:22.865573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e7483-d4dd-470d-8020-2aa07b06644f service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:22.868734 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-959e7483-d4dd-470d-8020-2aa07b06644f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:22.894268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e7483-d4dd-470d-8020-2aa07b06644f service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:22.910524 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd583e-5704-4778-83d3-df178b2da22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:22.911021 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd583e-5704-4778-83d3-df178b2da22f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:22.911580 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd583e-5704-4778-83d3-df178b2da22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:22.912085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd583e-5704-4778-83d3-df178b2da22f admin admin] PATH_INFO: `/v3/projects/97caa6530ed54751bcda16beeb647672` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:22.919243 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd583e-5704-4778-83d3-df178b2da22f admin admin] RBAC: Authorizing `identity:delete_project(project_id=97caa6530ed54751bcda16beeb647672)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:22.922731 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd583e-5704-4778-83d3-df178b2da22f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:22.948705 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-55e541cb-0a65-4426-a44a-b6f978b2d9d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc62214a1f4043209d2c6d0c88c2b058'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:22.949499 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-55e541cb-0a65-4426-a44a-b6f978b2d9d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc62214a1f4043209d2c6d0c88c2b058'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:22.951013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-55e541cb-0a65-4426-a44a-b6f978b2d9d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc62214a1f4043209d2c6d0c88c2b058'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:22.971713 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 944/1887] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:22 2022] DELETE /identity/v3/users/bc62214a1f4043209d2c6d0c88c2b058 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:22.976966 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff7e69-3c83-4b37-8442-2d76f6533808 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:22.977259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff7e69-3c83-4b37-8442-2d76f6533808 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:22.977543 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff7e69-3c83-4b37-8442-2d76f6533808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:22.998523 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-42ff7e69-3c83-4b37-8442-2d76f6533808 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:22.999779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-42ff7e69-3c83-4b37-8442-2d76f6533808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:23.026496 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:23.089021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 945/1888] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:18:23.096510 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff7e69-3c83-4b37-8442-2d76f6533808 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:23.100622 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-42ff7e69-3c83-4b37-8442-2d76f6533808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:23.103080 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6ffd583e-5704-4778-83d3-df178b2da22f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97caa6530ed54751bcda16beeb647672', 'request_id': 'req-6ffd583e-5704-4778-83d3-df178b2da22f'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:23.113349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6ffd583e-5704-4778-83d3-df178b2da22f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97caa6530ed54751bcda16beeb647672', 'request_id': 'req-6ffd583e-5704-4778-83d3-df178b2da22f'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:23.123603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6ffd583e-5704-4778-83d3-df178b2da22f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97caa6530ed54751bcda16beeb647672', 'request_id': 'req-6ffd583e-5704-4778-83d3-df178b2da22f'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:23.189217 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff7e69-3c83-4b37-8442-2d76f6533808 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:23.218711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6ffd583e-5704-4778-83d3-df178b2da22f admin admin] The token cache is being invalidate because project 97caa6530ed54751bcda16beeb647672 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:23.219338 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6ffd583e-5704-4778-83d3-df178b2da22f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:23.220152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6ffd583e-5704-4778-83d3-df178b2da22f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:23.225319 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 944/1889] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:22 2022] DELETE /identity/v3/projects/97caa6530ed54751bcda16beeb647672 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:23.227293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800b5c4-d11e-4255-a3e2-ffa1cd5b88de None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:23.227744 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800b5c4-d11e-4255-a3e2-ffa1cd5b88de None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:23.228349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7800b5c4-d11e-4255-a3e2-ffa1cd5b88de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:23.251899 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7800b5c4-d11e-4255-a3e2-ffa1cd5b88de None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:23.253770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7800b5c4-d11e-4255-a3e2-ffa1cd5b88de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:23.262647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba9457-e946-486c-9d52-3487c18cd9bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:23.264145 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba9457-e946-486c-9d52-3487c18cd9bc service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:23.264431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba9457-e946-486c-9d52-3487c18cd9bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:23.264709 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba9457-e946-486c-9d52-3487c18cd9bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:23.266825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-59ba9457-e946-486c-9d52-3487c18cd9bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:23.342952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 945/1890] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:23.353124 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a1fa5-6543-4c0f-9f16-ce7c5f7dd1c3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:23.353441 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a1fa5-6543-4c0f-9f16-ce7c5f7dd1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:23.353746 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a1fa5-6543-4c0f-9f16-ce7c5f7dd1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:23.369560 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba9457-e946-486c-9d52-3487c18cd9bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:23.371730 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ba9457-e946-486c-9d52-3487c18cd9bc service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:23.375521 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9e6a1fa5-6543-4c0f-9f16-ce7c5f7dd1c3 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:23.376751 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9e6a1fa5-6543-4c0f-9f16-ce7c5f7dd1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:23.388013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 946/1891] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:18:23.388773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 946/1892] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:23.392917 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba9457-e946-486c-9d52-3487c18cd9bc service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:23.410195 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba9457-e946-486c-9d52-3487c18cd9bc service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:23.415006 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e767be3-081c-45d1-8ccd-84075c375e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:23.415918 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e767be3-081c-45d1-8ccd-84075c375e22 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:23.416564 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e767be3-081c-45d1-8ccd-84075c375e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:23.416564 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e767be3-081c-45d1-8ccd-84075c375e22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:23.417441 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e767be3-081c-45d1-8ccd-84075c375e22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:18:23.417814 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e767be3-081c-45d1-8ccd-84075c375e22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:23.419727 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e767be3-081c-45d1-8ccd-84075c375e22 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:23.432487 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 947/1893] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:18:23.437515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a1fa5-6543-4c0f-9f16-ce7c5f7dd1c3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:23.449369 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a1fa5-6543-4c0f-9f16-ce7c5f7dd1c3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:23.454992 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9cfec-c4a6-43ec-844b-1a655c9ee3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:23.455940 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9cfec-c4a6-43ec-844b-1a655c9ee3fa admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:23.456334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9cfec-c4a6-43ec-844b-1a655c9ee3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:23.456703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9cfec-c4a6-43ec-844b-1a655c9ee3fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:23.457511 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9cfec-c4a6-43ec-844b-1a655c9ee3fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:23.459980 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9cfec-c4a6-43ec-844b-1a655c9ee3fa admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:23.553334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 947/1894] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:18:23 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:23.557181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e767be3-081c-45d1-8ccd-84075c375e22 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:23.573133 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e767be3-081c-45d1-8ccd-84075c375e22 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:23.662596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be485fd-85c3-489d-98af-d89429257778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:23.663247 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be485fd-85c3-489d-98af-d89429257778 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:23.663536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be485fd-85c3-489d-98af-d89429257778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:23.663805 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be485fd-85c3-489d-98af-d89429257778 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:23.664510 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be485fd-85c3-489d-98af-d89429257778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:23.666490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be485fd-85c3-489d-98af-d89429257778 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:23.754610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 948/1895] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:18:23 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:23.758745 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9cfec-c4a6-43ec-844b-1a655c9ee3fa admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:23.770101 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9cfec-c4a6-43ec-844b-1a655c9ee3fa admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:23.774891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528ca29-a722-45b2-9998-6df957b94a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:23.775551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528ca29-a722-45b2-9998-6df957b94a73 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:23.775837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528ca29-a722-45b2-9998-6df957b94a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:23.776114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528ca29-a722-45b2-9998-6df957b94a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:23.776866 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528ca29-a722-45b2-9998-6df957b94a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:23.779258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528ca29-a722-45b2-9998-6df957b94a73 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:23.789527 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 948/1896] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:23.794552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be485fd-85c3-489d-98af-d89429257778 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:23.804963 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be485fd-85c3-489d-98af-d89429257778 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:23.810063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523183e-f419-41d4-a5d0-89dea676e35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:23.810765 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523183e-f419-41d4-a5d0-89dea676e35a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:23.811058 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523183e-f419-41d4-a5d0-89dea676e35a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:23.811377 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523183e-f419-41d4-a5d0-89dea676e35a admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:23.812102 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523183e-f419-41d4-a5d0-89dea676e35a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:23.813728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523183e-f419-41d4-a5d0-89dea676e35a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:23.824114 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 949/1897] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:23.828196 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528ca29-a722-45b2-9998-6df957b94a73 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:23.840571 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528ca29-a722-45b2-9998-6df957b94a73 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:23.849386 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3bed1-4b64-4bfc-9b98-d904d0a4bdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:23.849386 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3bed1-4b64-4bfc-9b98-d904d0a4bdd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:23.849386 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3bed1-4b64-4bfc-9b98-d904d0a4bdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:23.849386 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3bed1-4b64-4bfc-9b98-d904d0a4bdd3 admin admin] PATH_INFO: `/v3/projects/09b6462b30a34629bda3cfb3b7270c6d/users/9dbe4952ee034749b771ff380da6e06c/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:23.853418 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523183e-f419-41d4-a5d0-89dea676e35a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:23.855811 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8523183e-f419-41d4-a5d0-89dea676e35a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:23.873448 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523183e-f419-41d4-a5d0-89dea676e35a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:23.879540 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74a573-995e-4a51-a901-f8a939ececb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:23.880558 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74a573-995e-4a51-a901-f8a939ececb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:23.880909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74a573-995e-4a51-a901-f8a939ececb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:23.881201 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74a573-995e-4a51-a901-f8a939ececb4 admin admin] PATH_INFO: `/v3/projects/9597f37257e640b095fae407357fa2b9` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:23.883800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3bed1-4b64-4bfc-9b98-d904d0a4bdd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b6462b30a34629bda3cfb3b7270c6d, user_id=9dbe4952ee034749b771ff380da6e06c, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:23.884246 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74a573-995e-4a51-a901-f8a939ececb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9597f37257e640b095fae407357fa2b9)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:23.884360 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74a573-995e-4a51-a901-f8a939ececb4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:23.885106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f3bed1-4b64-4bfc-9b98-d904d0a4bdd3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:23.971579 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 949/1898] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:23 2022] PUT /identity/v3/projects/09b6462b30a34629bda3cfb3b7270c6d/users/9dbe4952ee034749b771ff380da6e06c/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:23.976479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f95a0-9202-4b2c-8d6f-477c239d11d1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:23.976750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f95a0-9202-4b2c-8d6f-477c239d11d1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:23.977031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f95a0-9202-4b2c-8d6f-477c239d11d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:24.010216 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5d8f95a0-9202-4b2c-8d6f-477c239d11d1 None None] MFA Rules not processed for user `9dbe4952ee034749b771ff380da6e06c`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:24.011841 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5d8f95a0-9202-4b2c-8d6f-477c239d11d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:24.013460 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1a74a573-995e-4a51-a901-f8a939ececb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9597f37257e640b095fae407357fa2b9', 'request_id': 'req-1a74a573-995e-4a51-a901-f8a939ececb4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:24.091009 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 950/1899] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:18:24.095149 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1a74a573-995e-4a51-a901-f8a939ececb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9597f37257e640b095fae407357fa2b9', 'request_id': 'req-1a74a573-995e-4a51-a901-f8a939ececb4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:24.101784 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1a74a573-995e-4a51-a901-f8a939ececb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9597f37257e640b095fae407357fa2b9', 'request_id': 'req-1a74a573-995e-4a51-a901-f8a939ececb4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:24.102356 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f95a0-9202-4b2c-8d6f-477c239d11d1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:24.105108 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5d8f95a0-9202-4b2c-8d6f-477c239d11d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:24.117108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1a74a573-995e-4a51-a901-f8a939ececb4 admin admin] The token cache is being invalidate because project 9597f37257e640b095fae407357fa2b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:24.117406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1a74a573-995e-4a51-a901-f8a939ececb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:24.119159 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1a74a573-995e-4a51-a901-f8a939ececb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:24.123230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 950/1900] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:23 2022] DELETE /identity/v3/projects/9597f37257e640b095fae407357fa2b9 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:18:24.127282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74a573-995e-4a51-a901-f8a939ececb4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:24.129166 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1a74a573-995e-4a51-a901-f8a939ececb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:24.175047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f95a0-9202-4b2c-8d6f-477c239d11d1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:24.178880 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c6a8e-0d54-44c8-9c59-fed420072f50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:24.179433 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c6a8e-0d54-44c8-9c59-fed420072f50 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:24.179714 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c6a8e-0d54-44c8-9c59-fed420072f50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:24.179991 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c6a8e-0d54-44c8-9c59-fed420072f50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:24.186391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:24.190212 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0c1c6a8e-0d54-44c8-9c59-fed420072f50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:24.203041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c6a8e-0d54-44c8-9c59-fed420072f50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:24.204482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1c6a8e-0d54-44c8-9c59-fed420072f50 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:24.212091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74a573-995e-4a51-a901-f8a939ececb4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:24.214643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 951/1901] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:24.217605 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bce3a4-097c-4136-8c36-1fd3155cf32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:24.218530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bce3a4-097c-4136-8c36-1fd3155cf32d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:24.218814 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bce3a4-097c-4136-8c36-1fd3155cf32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:24.219157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bce3a4-097c-4136-8c36-1fd3155cf32d admin admin] PATH_INFO: `/v3/users/93dcc4168895488fb19d9038b5741420` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:24.220293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bce3a4-097c-4136-8c36-1fd3155cf32d admin admin] RBAC: Authorizing `identity:delete_user(user_id=93dcc4168895488fb19d9038b5741420)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:24.222270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bce3a4-097c-4136-8c36-1fd3155cf32d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:24.365617 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:24.501981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a2bce3a4-097c-4136-8c36-1fd3155cf32d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93dcc4168895488fb19d9038b5741420'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:24.517397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a2bce3a4-097c-4136-8c36-1fd3155cf32d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93dcc4168895488fb19d9038b5741420'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:24.578125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a2bce3a4-097c-4136-8c36-1fd3155cf32d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93dcc4168895488fb19d9038b5741420'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:24.578917 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a2bce3a4-097c-4136-8c36-1fd3155cf32d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93dcc4168895488fb19d9038b5741420'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:24.579762 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a2bce3a4-097c-4136-8c36-1fd3155cf32d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93dcc4168895488fb19d9038b5741420'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:24.609890 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 951/1902] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:24 2022] DELETE /identity/v3/users/93dcc4168895488fb19d9038b5741420 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:18:25.108730 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:25.179819 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c6a8e-0d54-44c8-9c59-fed420072f50 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:25.210184 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1c6a8e-0d54-44c8-9c59-fed420072f50 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:25.343391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3574f0a-b4f8-44ca-8c2c-3612f4e3cd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:25.344525 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3574f0a-b4f8-44ca-8c2c-3612f4e3cd0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:25.344987 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3574f0a-b4f8-44ca-8c2c-3612f4e3cd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:25.345426 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3574f0a-b4f8-44ca-8c2c-3612f4e3cd0f admin admin] PATH_INFO: `/v3/projects/9dfea382aea84994924e5110ed6757ec` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:25.347060 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3574f0a-b4f8-44ca-8c2c-3612f4e3cd0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dfea382aea84994924e5110ed6757ec)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:25.357047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3574f0a-b4f8-44ca-8c2c-3612f4e3cd0f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:25.539829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c3574f0a-b4f8-44ca-8c2c-3612f4e3cd0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dfea382aea84994924e5110ed6757ec', 'request_id': 'req-c3574f0a-b4f8-44ca-8c2c-3612f4e3cd0f'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:25.548995 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c3574f0a-b4f8-44ca-8c2c-3612f4e3cd0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dfea382aea84994924e5110ed6757ec', 'request_id': 'req-c3574f0a-b4f8-44ca-8c2c-3612f4e3cd0f'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:25.563446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c3574f0a-b4f8-44ca-8c2c-3612f4e3cd0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dfea382aea84994924e5110ed6757ec', 'request_id': 'req-c3574f0a-b4f8-44ca-8c2c-3612f4e3cd0f'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:25.672251 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:25.679245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c3574f0a-b4f8-44ca-8c2c-3612f4e3cd0f admin admin] The token cache is being invalidate because project 9dfea382aea84994924e5110ed6757ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:25.679753 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c3574f0a-b4f8-44ca-8c2c-3612f4e3cd0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:25.680403 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c3574f0a-b4f8-44ca-8c2c-3612f4e3cd0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:25.682048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 952/1903] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:25 2022] DELETE /identity/v3/projects/9dfea382aea84994924e5110ed6757ec => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:25.686118 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bce3a4-097c-4136-8c36-1fd3155cf32d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:25.687557 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a2bce3a4-097c-4136-8c36-1fd3155cf32d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:25.702218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bce3a4-097c-4136-8c36-1fd3155cf32d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:25.764165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6f4c3-9247-4024-83bc-15efb504b96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:25.766153 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6f4c3-9247-4024-83bc-15efb504b96d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:25.766484 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6f4c3-9247-4024-83bc-15efb504b96d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:25.766787 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b6f4c3-9247-4024-83bc-15efb504b96d admin admin] PATH_INFO: `/v3/users/bd883844c8a74d29ad92d1fe45bb870e` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:25.768546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6f4c3-9247-4024-83bc-15efb504b96d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd883844c8a74d29ad92d1fe45bb870e)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:25.770638 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b6f4c3-9247-4024-83bc-15efb504b96d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:25.974115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d3b6f4c3-9247-4024-83bc-15efb504b96d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd883844c8a74d29ad92d1fe45bb870e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:25.986569 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d3b6f4c3-9247-4024-83bc-15efb504b96d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd883844c8a74d29ad92d1fe45bb870e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:26.062024 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d3b6f4c3-9247-4024-83bc-15efb504b96d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd883844c8a74d29ad92d1fe45bb870e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:26.062764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d3b6f4c3-9247-4024-83bc-15efb504b96d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd883844c8a74d29ad92d1fe45bb870e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:26.063442 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d3b6f4c3-9247-4024-83bc-15efb504b96d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd883844c8a74d29ad92d1fe45bb870e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:26.081627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 952/1904] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:25 2022] DELETE /identity/v3/users/bd883844c8a74d29ad92d1fe45bb870e => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:26.200605 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5de734-e49f-4c04-97e5-410e9897546b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:26.200605 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5de734-e49f-4c04-97e5-410e9897546b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:26.200605 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5de734-e49f-4c04-97e5-410e9897546b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:26.232932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-db5de734-e49f-4c04-97e5-410e9897546b None None] MFA Rules not processed for user `9dbe4952ee034749b771ff380da6e06c`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:26.234398 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-db5de734-e49f-4c04-97e5-410e9897546b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:26.365720 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 953/1905] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:27.217044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6f4c3-9247-4024-83bc-15efb504b96d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:27.218661 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d3b6f4c3-9247-4024-83bc-15efb504b96d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:27.271037 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:27.331824 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b6f4c3-9247-4024-83bc-15efb504b96d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:27.337051 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cb4f3-7f99-4947-b1f0-d0264be904f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:27.337944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cb4f3-7f99-4947-b1f0-d0264be904f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:27.338234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cb4f3-7f99-4947-b1f0-d0264be904f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:27.338584 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cb4f3-7f99-4947-b1f0-d0264be904f3 admin admin] PATH_INFO: `/v3/projects/e110cf156df446e2b2c342181f177651` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:27.339817 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cb4f3-7f99-4947-b1f0-d0264be904f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e110cf156df446e2b2c342181f177651)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:27.342471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52cb4f3-7f99-4947-b1f0-d0264be904f3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:27.483261 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b52cb4f3-7f99-4947-b1f0-d0264be904f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e110cf156df446e2b2c342181f177651', 'request_id': 'req-b52cb4f3-7f99-4947-b1f0-d0264be904f3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:27.557949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b52cb4f3-7f99-4947-b1f0-d0264be904f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e110cf156df446e2b2c342181f177651', 'request_id': 'req-b52cb4f3-7f99-4947-b1f0-d0264be904f3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:27.575946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b52cb4f3-7f99-4947-b1f0-d0264be904f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e110cf156df446e2b2c342181f177651', 'request_id': 'req-b52cb4f3-7f99-4947-b1f0-d0264be904f3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:27.610255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b52cb4f3-7f99-4947-b1f0-d0264be904f3 admin admin] The token cache is being invalidate because project e110cf156df446e2b2c342181f177651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:27.610596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b52cb4f3-7f99-4947-b1f0-d0264be904f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:27.611660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b52cb4f3-7f99-4947-b1f0-d0264be904f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:27.614299 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 953/1906] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:27 2022] DELETE /identity/v3/projects/e110cf156df446e2b2c342181f177651 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:27.621745 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c748cf-0e5f-44e3-be95-b068e611aed6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:27.622255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c748cf-0e5f-44e3-be95-b068e611aed6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:27.622353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c748cf-0e5f-44e3-be95-b068e611aed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:27.659258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-55c748cf-0e5f-44e3-be95-b068e611aed6 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:27.665244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-55c748cf-0e5f-44e3-be95-b068e611aed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:27.786511 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 954/1907] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:27.795156 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd574459-a7d0-4efa-89c7-34b4b871a59b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:27.795782 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd574459-a7d0-4efa-89c7-34b4b871a59b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:27.798265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd574459-a7d0-4efa-89c7-34b4b871a59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:27.817634 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-cd574459-a7d0-4efa-89c7-34b4b871a59b None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:27.818968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-cd574459-a7d0-4efa-89c7-34b4b871a59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:27.854925 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 954/1908] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:27.856588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c748cf-0e5f-44e3-be95-b068e611aed6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:27.875911 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c748cf-0e5f-44e3-be95-b068e611aed6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:27.883615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f23da-1662-4d55-aee8-c2a1ab75af16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:27.884554 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f23da-1662-4d55-aee8-c2a1ab75af16 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:27.884873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f23da-1662-4d55-aee8-c2a1ab75af16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:27.885333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f23da-1662-4d55-aee8-c2a1ab75af16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:27.886078 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f23da-1662-4d55-aee8-c2a1ab75af16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:18:27.886703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f23da-1662-4d55-aee8-c2a1ab75af16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:27.890055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f23da-1662-4d55-aee8-c2a1ab75af16 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:27.910226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 955/1909] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:18:27.915210 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd574459-a7d0-4efa-89c7-34b4b871a59b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:27.928785 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd574459-a7d0-4efa-89c7-34b4b871a59b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:27.944050 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef70b2-f765-4e1d-b6e9-5702406d9800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:27.944760 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef70b2-f765-4e1d-b6e9-5702406d9800 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:27.945043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef70b2-f765-4e1d-b6e9-5702406d9800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:27.945326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef70b2-f765-4e1d-b6e9-5702406d9800 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:27.946032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef70b2-f765-4e1d-b6e9-5702406d9800 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:27.948346 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef70b2-f765-4e1d-b6e9-5702406d9800 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:28.113120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 955/1910] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:18:27 2022] POST /identity/v3/projects => generated 381 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:28.113602 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f23da-1662-4d55-aee8-c2a1ab75af16 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:28.128869 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f23da-1662-4d55-aee8-c2a1ab75af16 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:28.241513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a4a84-2c97-40f5-b227-cc1194cdd0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:28.243560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a4a84-2c97-40f5-b227-cc1194cdd0c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:28.244047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a4a84-2c97-40f5-b227-cc1194cdd0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:28.251413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a4a84-2c97-40f5-b227-cc1194cdd0c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:28.251413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a4a84-2c97-40f5-b227-cc1194cdd0c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:28.251413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a4a84-2c97-40f5-b227-cc1194cdd0c8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:28.256351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:28.423503 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 956/1911] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:18:28 2022] POST /identity/v3/users => generated 291 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:18:28.424640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef70b2-f765-4e1d-b6e9-5702406d9800 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:28.436968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef70b2-f765-4e1d-b6e9-5702406d9800 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:28.441823 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e240b15-3e6a-4bff-92c6-f8ed699b7d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:28.442601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e240b15-3e6a-4bff-92c6-f8ed699b7d9d admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:28.442949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e240b15-3e6a-4bff-92c6-f8ed699b7d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:28.443290 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e240b15-3e6a-4bff-92c6-f8ed699b7d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:28.444087 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e240b15-3e6a-4bff-92c6-f8ed699b7d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:28.450941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e240b15-3e6a-4bff-92c6-f8ed699b7d9d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:28.460152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 956/1912] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:18:28.464134 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a4a84-2c97-40f5-b227-cc1194cdd0c8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:28.475632 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a4a84-2c97-40f5-b227-cc1194cdd0c8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:28.480378 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c4066d-9fcb-4df5-b010-873bc31c0ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:28.483862 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4066d-9fcb-4df5-b010-873bc31c0ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:28.483862 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4066d-9fcb-4df5-b010-873bc31c0ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:28.483862 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c4066d-9fcb-4df5-b010-873bc31c0ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:28.483862 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4066d-9fcb-4df5-b010-873bc31c0ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:28.484764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c4066d-9fcb-4df5-b010-873bc31c0ca3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:28.503262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 957/1913] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:28.504514 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e240b15-3e6a-4bff-92c6-f8ed699b7d9d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:28.515679 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e240b15-3e6a-4bff-92c6-f8ed699b7d9d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:28.520315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bc4a4-cd82-4c92-8973-59f1c8cc5b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:28.521279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bc4a4-cd82-4c92-8973-59f1c8cc5b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:28.521902 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bc4a4-cd82-4c92-8973-59f1c8cc5b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:28.522156 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130bc4a4-cd82-4c92-8973-59f1c8cc5b15 admin admin] PATH_INFO: `/v3/projects/407751e8a4354460bbe697115817c583/users/f84505b9b4f3491a875665c2ed325c4a/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:28.537203 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130bc4a4-cd82-4c92-8973-59f1c8cc5b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=407751e8a4354460bbe697115817c583, user_id=f84505b9b4f3491a875665c2ed325c4a, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:28.541162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130bc4a4-cd82-4c92-8973-59f1c8cc5b15 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:28.628609 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 957/1914] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:28 2022] PUT /identity/v3/projects/407751e8a4354460bbe697115817c583/users/f84505b9b4f3491a875665c2ed325c4a/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:28.643457 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78799aa-1862-42f7-ae6e-8535fe97c0bb None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:28.643990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78799aa-1862-42f7-ae6e-8535fe97c0bb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:28.644066 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78799aa-1862-42f7-ae6e-8535fe97c0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:28.678101 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d78799aa-1862-42f7-ae6e-8535fe97c0bb None None] MFA Rules not processed for user `f84505b9b4f3491a875665c2ed325c4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:28.679282 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d78799aa-1862-42f7-ae6e-8535fe97c0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:28.751933 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 958/1915] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:28.756479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bc4a4-cd82-4c92-8973-59f1c8cc5b15 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:28.821769 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:28.838886 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130bc4a4-cd82-4c92-8973-59f1c8cc5b15 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:28.844243 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c257e5d-1cbc-451a-af10-c67fe04ecb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:28.844911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c257e5d-1cbc-451a-af10-c67fe04ecb07 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:28.845200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c257e5d-1cbc-451a-af10-c67fe04ecb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:28.845476 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c257e5d-1cbc-451a-af10-c67fe04ecb07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:28.846170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c257e5d-1cbc-451a-af10-c67fe04ecb07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:28.848262 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c257e5d-1cbc-451a-af10-c67fe04ecb07 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:28.961451 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 958/1916] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:18:28 2022] POST /identity/v3/projects => generated 381 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:28.962698 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78799aa-1862-42f7-ae6e-8535fe97c0bb None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:28.974843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78799aa-1862-42f7-ae6e-8535fe97c0bb None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:29.085816 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b473e-8186-46b8-960a-648dfc055074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:29.086813 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b473e-8186-46b8-960a-648dfc055074 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:29.087248 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b473e-8186-46b8-960a-648dfc055074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:29.087672 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b473e-8186-46b8-960a-648dfc055074 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:29.088604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b473e-8186-46b8-960a-648dfc055074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:29.091443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b473e-8186-46b8-960a-648dfc055074 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:29.193422 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 959/1917] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:18:28 2022] POST /identity/v3/users => generated 297 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:29.198271 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c257e5d-1cbc-451a-af10-c67fe04ecb07 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:29.213283 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c257e5d-1cbc-451a-af10-c67fe04ecb07 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:29.219490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c66a0-1967-4d9f-be79-37f11dfa2209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:29.220204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c66a0-1967-4d9f-be79-37f11dfa2209 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:29.220490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c66a0-1967-4d9f-be79-37f11dfa2209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:29.221394 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c66a0-1967-4d9f-be79-37f11dfa2209 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:29.221786 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c66a0-1967-4d9f-be79-37f11dfa2209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:29.223431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c66a0-1967-4d9f-be79-37f11dfa2209 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:29.238481 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 959/1918] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:29.242572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b473e-8186-46b8-960a-648dfc055074 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:29.255045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b473e-8186-46b8-960a-648dfc055074 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:29.266731 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c54795-0815-486b-8a5b-29cfec533177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:29.269000 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c54795-0815-486b-8a5b-29cfec533177 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:29.269405 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c54795-0815-486b-8a5b-29cfec533177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:29.269761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c54795-0815-486b-8a5b-29cfec533177 admin admin] PATH_INFO: `/v3/projects/34f0b024a3ae4004b0c6bcadbb3bd5cb/users/795530f1e25a434bbdd4562c9db7ec64/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:29.311309 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c54795-0815-486b-8a5b-29cfec533177 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f0b024a3ae4004b0c6bcadbb3bd5cb, user_id=795530f1e25a434bbdd4562c9db7ec64, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:29.314494 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c54795-0815-486b-8a5b-29cfec533177 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:29.418693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 960/1919] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:29 2022] PUT /identity/v3/projects/34f0b024a3ae4004b0c6bcadbb3bd5cb/users/795530f1e25a434bbdd4562c9db7ec64/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:29.419851 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c66a0-1967-4d9f-be79-37f11dfa2209 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:29.433183 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c66a0-1967-4d9f-be79-37f11dfa2209 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:29.514051 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6689e-6aee-4a16-9cda-af991d09051a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:29.515155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6689e-6aee-4a16-9cda-af991d09051a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:29.515750 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6689e-6aee-4a16-9cda-af991d09051a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:29.516210 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6689e-6aee-4a16-9cda-af991d09051a admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:29.516997 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e6689e-6aee-4a16-9cda-af991d09051a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:29.518861 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e6689e-6aee-4a16-9cda-af991d09051a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:29.530491 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 960/1920] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:18:29.534900 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c54795-0815-486b-8a5b-29cfec533177 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:29.546771 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c54795-0815-486b-8a5b-29cfec533177 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:29.553565 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e3b7d-4f66-4ae6-b6d6-38a1306f65a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:29.554902 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e3b7d-4f66-4ae6-b6d6-38a1306f65a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:29.554902 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e3b7d-4f66-4ae6-b6d6-38a1306f65a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:29.555082 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e3b7d-4f66-4ae6-b6d6-38a1306f65a2 admin admin] PATH_INFO: `/v3/projects/34f0b024a3ae4004b0c6bcadbb3bd5cb/users/795530f1e25a434bbdd4562c9db7ec64/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:29.557083 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e3b7d-4f66-4ae6-b6d6-38a1306f65a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34f0b024a3ae4004b0c6bcadbb3bd5cb, user_id=795530f1e25a434bbdd4562c9db7ec64, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:29.560519 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7e3b7d-4f66-4ae6-b6d6-38a1306f65a2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:29.571274 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 961/1921] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:29 2022] PUT /identity/v3/projects/34f0b024a3ae4004b0c6bcadbb3bd5cb/users/795530f1e25a434bbdd4562c9db7ec64/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:29.585207 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5e561-45d6-4601-b5ae-78ed8bfae492 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:29.585506 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5e561-45d6-4601-b5ae-78ed8bfae492 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:29.585881 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5e561-45d6-4601-b5ae-78ed8bfae492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:29.616740 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-eda5e561-45d6-4601-b5ae-78ed8bfae492 None None] MFA Rules not processed for user `795530f1e25a434bbdd4562c9db7ec64`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:29.618686 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-eda5e561-45d6-4601-b5ae-78ed8bfae492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:29.709454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 961/1922] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:29.724805 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e3b7d-4f66-4ae6-b6d6-38a1306f65a2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:29.727111 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-db7e3b7d-4f66-4ae6-b6d6-38a1306f65a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:29.801567 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:29.859618 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e3b7d-4f66-4ae6-b6d6-38a1306f65a2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:29.864194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffecd5-fe8f-4ac3-980d-523f7bec68e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:29.864956 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffecd5-fe8f-4ac3-980d-523f7bec68e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:29.865412 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffecd5-fe8f-4ac3-980d-523f7bec68e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:29.865757 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffecd5-fe8f-4ac3-980d-523f7bec68e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:29.878806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffecd5-fe8f-4ac3-980d-523f7bec68e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:29.894223 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffecd5-fe8f-4ac3-980d-523f7bec68e5 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:29.905932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 962/1923] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:30.107043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5e561-45d6-4601-b5ae-78ed8bfae492 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:30.116869 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5e561-45d6-4601-b5ae-78ed8bfae492 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:30.120458 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5379e51b-7755-4458-8e41-c7a44648567c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:30.121127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379e51b-7755-4458-8e41-c7a44648567c service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:30.123335 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379e51b-7755-4458-8e41-c7a44648567c service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:30.123335 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5379e51b-7755-4458-8e41-c7a44648567c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:30.164580 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:30.200003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5379e51b-7755-4458-8e41-c7a44648567c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:30.200003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5379e51b-7755-4458-8e41-c7a44648567c service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:30.200364 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 962/1924] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:30.939649 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffecd5-fe8f-4ac3-980d-523f7bec68e5 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:31.005222 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffecd5-fe8f-4ac3-980d-523f7bec68e5 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:31.009761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261d6fc-7264-409d-93a9-af3e83b45cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:31.014353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261d6fc-7264-409d-93a9-af3e83b45cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:31.014353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261d6fc-7264-409d-93a9-af3e83b45cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:31.014353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5261d6fc-7264-409d-93a9-af3e83b45cc8 admin admin] PATH_INFO: `/v3/users/f84505b9b4f3491a875665c2ed325c4a` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:31.014353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5261d6fc-7264-409d-93a9-af3e83b45cc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f84505b9b4f3491a875665c2ed325c4a)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:31.015398 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5261d6fc-7264-409d-93a9-af3e83b45cc8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:31.213600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5261d6fc-7264-409d-93a9-af3e83b45cc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f84505b9b4f3491a875665c2ed325c4a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:31.225384 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5261d6fc-7264-409d-93a9-af3e83b45cc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f84505b9b4f3491a875665c2ed325c4a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:31.226053 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5261d6fc-7264-409d-93a9-af3e83b45cc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f84505b9b4f3491a875665c2ed325c4a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:31.226678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5261d6fc-7264-409d-93a9-af3e83b45cc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f84505b9b4f3491a875665c2ed325c4a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:31.233693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5261d6fc-7264-409d-93a9-af3e83b45cc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f84505b9b4f3491a875665c2ed325c4a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:31.306960 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 963/1925] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:30 2022] DELETE /identity/v3/users/f84505b9b4f3491a875665c2ed325c4a => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:31.311422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bb8aa-17f4-49ee-8386-39ef74ff3a1c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:31.311907 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bb8aa-17f4-49ee-8386-39ef74ff3a1c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:31.312004 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bb8aa-17f4-49ee-8386-39ef74ff3a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:31.328795 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-220bb8aa-17f4-49ee-8386-39ef74ff3a1c None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:31.330000 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-220bb8aa-17f4-49ee-8386-39ef74ff3a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:31.393272 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 963/1926] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:31.405569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261d6fc-7264-409d-93a9-af3e83b45cc8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:31.405569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5261d6fc-7264-409d-93a9-af3e83b45cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:31.454555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:31.468762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5261d6fc-7264-409d-93a9-af3e83b45cc8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:31.472417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ced324-4923-49c3-a0a1-65fffd01ff73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:31.473061 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ced324-4923-49c3-a0a1-65fffd01ff73 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:31.473344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ced324-4923-49c3-a0a1-65fffd01ff73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:31.473619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ced324-4923-49c3-a0a1-65fffd01ff73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:31.483849 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ced324-4923-49c3-a0a1-65fffd01ff73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:31.485743 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ced324-4923-49c3-a0a1-65fffd01ff73 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:31.495381 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 964/1927] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:32.049220 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bb8aa-17f4-49ee-8386-39ef74ff3a1c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:32.053849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-220bb8aa-17f4-49ee-8386-39ef74ff3a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:32.077655 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bb8aa-17f4-49ee-8386-39ef74ff3a1c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:32.087543 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b387565-51c6-4b30-a172-9ac61b55501b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:32.089084 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b387565-51c6-4b30-a172-9ac61b55501b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:32.089590 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b387565-51c6-4b30-a172-9ac61b55501b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:32.090673 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b387565-51c6-4b30-a172-9ac61b55501b admin admin] PATH_INFO: `/v3/projects/407751e8a4354460bbe697115817c583` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:32.092888 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b387565-51c6-4b30-a172-9ac61b55501b admin admin] RBAC: Authorizing `identity:delete_project(project_id=407751e8a4354460bbe697115817c583)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:32.098384 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b387565-51c6-4b30-a172-9ac61b55501b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:32.251069 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5b387565-51c6-4b30-a172-9ac61b55501b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '407751e8a4354460bbe697115817c583', 'request_id': 'req-5b387565-51c6-4b30-a172-9ac61b55501b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:32.317107 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5b387565-51c6-4b30-a172-9ac61b55501b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '407751e8a4354460bbe697115817c583', 'request_id': 'req-5b387565-51c6-4b30-a172-9ac61b55501b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:32.323161 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5b387565-51c6-4b30-a172-9ac61b55501b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '407751e8a4354460bbe697115817c583', 'request_id': 'req-5b387565-51c6-4b30-a172-9ac61b55501b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:32.334522 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5b387565-51c6-4b30-a172-9ac61b55501b admin admin] The token cache is being invalidate because project 407751e8a4354460bbe697115817c583 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:32.334813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5b387565-51c6-4b30-a172-9ac61b55501b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:32.335471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5b387565-51c6-4b30-a172-9ac61b55501b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:32.337034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 964/1928] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:32 2022] DELETE /identity/v3/projects/407751e8a4354460bbe697115817c583 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:32.341615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ced324-4923-49c3-a0a1-65fffd01ff73 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:32.360252 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ced324-4923-49c3-a0a1-65fffd01ff73 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:32.420484 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62cd22-13e9-4a7e-bbce-e66c29f3137c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:32.427700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62cd22-13e9-4a7e-bbce-e66c29f3137c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:32.427700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62cd22-13e9-4a7e-bbce-e66c29f3137c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:32.427700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62cd22-13e9-4a7e-bbce-e66c29f3137c admin admin] PATH_INFO: `/v3/users/795530f1e25a434bbdd4562c9db7ec64` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:32.427700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62cd22-13e9-4a7e-bbce-e66c29f3137c admin admin] RBAC: Authorizing `identity:delete_user(user_id=795530f1e25a434bbdd4562c9db7ec64)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:32.427700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d62cd22-13e9-4a7e-bbce-e66c29f3137c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:32.633324 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3d62cd22-13e9-4a7e-bbce-e66c29f3137c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '795530f1e25a434bbdd4562c9db7ec64'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:32.646226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3d62cd22-13e9-4a7e-bbce-e66c29f3137c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '795530f1e25a434bbdd4562c9db7ec64'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:32.647224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3d62cd22-13e9-4a7e-bbce-e66c29f3137c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '795530f1e25a434bbdd4562c9db7ec64'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:32.650458 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3d62cd22-13e9-4a7e-bbce-e66c29f3137c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '795530f1e25a434bbdd4562c9db7ec64'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:32.655483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3d62cd22-13e9-4a7e-bbce-e66c29f3137c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '795530f1e25a434bbdd4562c9db7ec64'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:32.701308 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 965/1929] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:32 2022] DELETE /identity/v3/users/795530f1e25a434bbdd4562c9db7ec64 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:33.168917 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b387565-51c6-4b30-a172-9ac61b55501b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:33.171091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5b387565-51c6-4b30-a172-9ac61b55501b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:33.198645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b387565-51c6-4b30-a172-9ac61b55501b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:33.259381 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd15eb-f79b-4a57-9dff-ff965d526882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:33.260284 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd15eb-f79b-4a57-9dff-ff965d526882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:33.260566 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd15eb-f79b-4a57-9dff-ff965d526882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:33.260841 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd15eb-f79b-4a57-9dff-ff965d526882 admin admin] PATH_INFO: `/v3/projects/34f0b024a3ae4004b0c6bcadbb3bd5cb` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:33.262331 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd15eb-f79b-4a57-9dff-ff965d526882 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34f0b024a3ae4004b0c6bcadbb3bd5cb)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:33.266301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd15eb-f79b-4a57-9dff-ff965d526882 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:33.268503 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:33.389168 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-73bd15eb-f79b-4a57-9dff-ff965d526882 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34f0b024a3ae4004b0c6bcadbb3bd5cb', 'request_id': 'req-73bd15eb-f79b-4a57-9dff-ff965d526882'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:33.480211 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-73bd15eb-f79b-4a57-9dff-ff965d526882 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34f0b024a3ae4004b0c6bcadbb3bd5cb', 'request_id': 'req-73bd15eb-f79b-4a57-9dff-ff965d526882'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:33.487566 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-73bd15eb-f79b-4a57-9dff-ff965d526882 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34f0b024a3ae4004b0c6bcadbb3bd5cb', 'request_id': 'req-73bd15eb-f79b-4a57-9dff-ff965d526882'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:33.499909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-73bd15eb-f79b-4a57-9dff-ff965d526882 admin admin] The token cache is being invalidate because project 34f0b024a3ae4004b0c6bcadbb3bd5cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:33.500194 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-73bd15eb-f79b-4a57-9dff-ff965d526882 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:33.500834 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-73bd15eb-f79b-4a57-9dff-ff965d526882 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:33.502575 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 965/1930] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:33 2022] DELETE /identity/v3/projects/34f0b024a3ae4004b0c6bcadbb3bd5cb => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:33.507886 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8937ffb-708c-4404-b608-7918934bf316 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:33.508372 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8937ffb-708c-4404-b608-7918934bf316 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:33.508477 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8937ffb-708c-4404-b608-7918934bf316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:33.527525 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f8937ffb-708c-4404-b608-7918934bf316 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:33.530456 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f8937ffb-708c-4404-b608-7918934bf316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:33.589069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 966/1931] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:33.597108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd134808-4d57-4817-a962-1dca26c6c662 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:33.597396 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd134808-4d57-4817-a962-1dca26c6c662 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:33.597719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd134808-4d57-4817-a962-1dca26c6c662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:33.617200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-fd134808-4d57-4817-a962-1dca26c6c662 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:33.620475 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fd134808-4d57-4817-a962-1dca26c6c662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:33.628595 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 966/1932] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:33.633016 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8937ffb-708c-4404-b608-7918934bf316 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:33.643447 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8937ffb-708c-4404-b608-7918934bf316 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:33.648381 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506ce06-233b-4569-b9f0-5393468ff4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:33.648990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506ce06-233b-4569-b9f0-5393468ff4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:33.649279 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506ce06-233b-4569-b9f0-5393468ff4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:33.649561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506ce06-233b-4569-b9f0-5393468ff4d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:33.650327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506ce06-233b-4569-b9f0-5393468ff4d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:18:33.650676 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506ce06-233b-4569-b9f0-5393468ff4d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:33.652283 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506ce06-233b-4569-b9f0-5393468ff4d3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:33.653754 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:33.664916 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 967/1933] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:18:33.670592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd134808-4d57-4817-a962-1dca26c6c662 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:33.679708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd134808-4d57-4817-a962-1dca26c6c662 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:33.684324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a2b86-77fe-44ac-a096-c56e76b88139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:33.684951 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a2b86-77fe-44ac-a096-c56e76b88139 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:33.685236 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a2b86-77fe-44ac-a096-c56e76b88139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:33.686223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a2b86-77fe-44ac-a096-c56e76b88139 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:33.687040 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a2b86-77fe-44ac-a096-c56e76b88139 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:33.689052 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a2b86-77fe-44ac-a096-c56e76b88139 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:33.784407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 967/1934] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:18:33 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:33.788714 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506ce06-233b-4569-b9f0-5393468ff4d3 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:33.798875 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506ce06-233b-4569-b9f0-5393468ff4d3 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:33.855175 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374021eb-ab8f-4d31-a5ea-7b02c5560fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:33.855840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374021eb-ab8f-4d31-a5ea-7b02c5560fa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:33.856166 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374021eb-ab8f-4d31-a5ea-7b02c5560fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:33.856444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374021eb-ab8f-4d31-a5ea-7b02c5560fa4 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:33.857202 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374021eb-ab8f-4d31-a5ea-7b02c5560fa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:33.859207 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374021eb-ab8f-4d31-a5ea-7b02c5560fa4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:33.963717 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 968/1935] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:18:33 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:18:33.968263 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a2b86-77fe-44ac-a096-c56e76b88139 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:33.982979 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a2b86-77fe-44ac-a096-c56e76b88139 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:33.984473 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a68d5-b5a8-44fc-850e-66906b8a519d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:33.984473 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a68d5-b5a8-44fc-850e-66906b8a519d admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:33.984671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a68d5-b5a8-44fc-850e-66906b8a519d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:33.984879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a68d5-b5a8-44fc-850e-66906b8a519d admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:33.985600 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a68d5-b5a8-44fc-850e-66906b8a519d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:33.987226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a68d5-b5a8-44fc-850e-66906b8a519d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:34.001039 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 968/1936] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:18:34.002257 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374021eb-ab8f-4d31-a5ea-7b02c5560fa4 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:34.017673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374021eb-ab8f-4d31-a5ea-7b02c5560fa4 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:34.022753 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb9c1d8-7bdf-4590-a84b-95fdbc8e1472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:34.023420 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9c1d8-7bdf-4590-a84b-95fdbc8e1472 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:34.023711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9c1d8-7bdf-4590-a84b-95fdbc8e1472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:34.023997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb9c1d8-7bdf-4590-a84b-95fdbc8e1472 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:34.024732 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb9c1d8-7bdf-4590-a84b-95fdbc8e1472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:34.026333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb9c1d8-7bdf-4590-a84b-95fdbc8e1472 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:34.037949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 969/1937] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:34.042327 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a68d5-b5a8-44fc-850e-66906b8a519d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:34.057611 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a68d5-b5a8-44fc-850e-66906b8a519d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:34.062942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2de830-ca46-4423-9c73-90b62d292f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:34.063827 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2de830-ca46-4423-9c73-90b62d292f2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:34.064125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2de830-ca46-4423-9c73-90b62d292f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:34.064377 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2de830-ca46-4423-9c73-90b62d292f2e admin admin] PATH_INFO: `/v3/projects/ce8010570e7f40bbafaf15b9330456c2/users/4ed59d5ecb8f461c8614865bc1d1da3b/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:34.079060 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2de830-ca46-4423-9c73-90b62d292f2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce8010570e7f40bbafaf15b9330456c2, user_id=4ed59d5ecb8f461c8614865bc1d1da3b, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:34.081792 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2de830-ca46-4423-9c73-90b62d292f2e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:34.186460 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 969/1938] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:34 2022] PUT /identity/v3/projects/ce8010570e7f40bbafaf15b9330456c2/users/4ed59d5ecb8f461c8614865bc1d1da3b/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:34.196509 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f64368-0bb8-42f3-9b64-4965dda3ff4f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:34.196799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f64368-0bb8-42f3-9b64-4965dda3ff4f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:34.197079 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f64368-0bb8-42f3-9b64-4965dda3ff4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:34.225723 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e1f64368-0bb8-42f3-9b64-4965dda3ff4f None None] MFA Rules not processed for user `4ed59d5ecb8f461c8614865bc1d1da3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:34.226964 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e1f64368-0bb8-42f3-9b64-4965dda3ff4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:34.308532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 970/1939] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:34.320139 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2de830-ca46-4423-9c73-90b62d292f2e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:34.321858 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7e2de830-ca46-4423-9c73-90b62d292f2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:34.389248 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2de830-ca46-4423-9c73-90b62d292f2e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:34.393156 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0de973-ee7b-4eaa-932d-b72409b52ab1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:34.393917 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0de973-ee7b-4eaa-932d-b72409b52ab1 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:34.394206 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0de973-ee7b-4eaa-932d-b72409b52ab1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:34.394518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0de973-ee7b-4eaa-932d-b72409b52ab1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:34.405693 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0de973-ee7b-4eaa-932d-b72409b52ab1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:34.407147 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0de973-ee7b-4eaa-932d-b72409b52ab1 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:34.416405 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 970/1940] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:37.676748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f64368-0bb8-42f3-9b64-4965dda3ff4f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:37.678471 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e1f64368-0bb8-42f3-9b64-4965dda3ff4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:37.733718 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0de973-ee7b-4eaa-932d-b72409b52ab1 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:37.756801 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f64368-0bb8-42f3-9b64-4965dda3ff4f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:37.761501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3581fc68-1bdf-4118-9eda-746ea9bf7593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:37.762402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581fc68-1bdf-4118-9eda-746ea9bf7593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:37.763022 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581fc68-1bdf-4118-9eda-746ea9bf7593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:37.763022 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3581fc68-1bdf-4118-9eda-746ea9bf7593 admin admin] PATH_INFO: `/v3/users/c3be6ae7a5fb415b93783dd7b0e79fe1` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:37.764763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581fc68-1bdf-4118-9eda-746ea9bf7593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3be6ae7a5fb415b93783dd7b0e79fe1)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:37.766738 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3581fc68-1bdf-4118-9eda-746ea9bf7593 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:37.769493 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:37.772979 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:37.804319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0de973-ee7b-4eaa-932d-b72409b52ab1 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:37.808998 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb1de9-21a7-4985-a584-df3cf131ff55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:37.810055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb1de9-21a7-4985-a584-df3cf131ff55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:37.811096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb1de9-21a7-4985-a584-df3cf131ff55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:37.811096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb1de9-21a7-4985-a584-df3cf131ff55 admin admin] PATH_INFO: `/v3/users/4ed59d5ecb8f461c8614865bc1d1da3b` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:37.812580 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb1de9-21a7-4985-a584-df3cf131ff55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ed59d5ecb8f461c8614865bc1d1da3b)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:37.814627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb1de9-21a7-4985-a584-df3cf131ff55 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:37.953778 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3581fc68-1bdf-4118-9eda-746ea9bf7593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3be6ae7a5fb415b93783dd7b0e79fe1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:37.966203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3581fc68-1bdf-4118-9eda-746ea9bf7593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3be6ae7a5fb415b93783dd7b0e79fe1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:37.968033 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3581fc68-1bdf-4118-9eda-746ea9bf7593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3be6ae7a5fb415b93783dd7b0e79fe1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:37.968645 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3581fc68-1bdf-4118-9eda-746ea9bf7593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3be6ae7a5fb415b93783dd7b0e79fe1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:37.975634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3581fc68-1bdf-4118-9eda-746ea9bf7593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3be6ae7a5fb415b93783dd7b0e79fe1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:38.037200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d9cb1de9-21a7-4985-a584-df3cf131ff55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ed59d5ecb8f461c8614865bc1d1da3b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:38.050652 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d9cb1de9-21a7-4985-a584-df3cf131ff55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ed59d5ecb8f461c8614865bc1d1da3b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:38.057025 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 971/1941] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:37 2022] DELETE /identity/v3/users/c3be6ae7a5fb415b93783dd7b0e79fe1 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:38.062262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753fbdb-46f3-4137-8454-18834666a071 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:38.062658 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753fbdb-46f3-4137-8454-18834666a071 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:38.063049 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753fbdb-46f3-4137-8454-18834666a071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:38.080556 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3753fbdb-46f3-4137-8454-18834666a071 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:38.081841 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3753fbdb-46f3-4137-8454-18834666a071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:38.146087 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 972/1942] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:38.153752 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d9cb1de9-21a7-4985-a584-df3cf131ff55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ed59d5ecb8f461c8614865bc1d1da3b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:38.154941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d9cb1de9-21a7-4985-a584-df3cf131ff55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ed59d5ecb8f461c8614865bc1d1da3b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:38.155869 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-d9cb1de9-21a7-4985-a584-df3cf131ff55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ed59d5ecb8f461c8614865bc1d1da3b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:38.156973 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753fbdb-46f3-4137-8454-18834666a071 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:38.158326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3753fbdb-46f3-4137-8454-18834666a071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:38.182216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 971/1943] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:37 2022] DELETE /identity/v3/users/4ed59d5ecb8f461c8614865bc1d1da3b => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:18:38.186740 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc0fc0-44ce-4a98-a8a3-411029aabcaf None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:38.187035 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc0fc0-44ce-4a98-a8a3-411029aabcaf None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:38.187317 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddc0fc0-44ce-4a98-a8a3-411029aabcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:38.205758 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4ddc0fc0-44ce-4a98-a8a3-411029aabcaf None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:38.207351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4ddc0fc0-44ce-4a98-a8a3-411029aabcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:38.231611 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753fbdb-46f3-4137-8454-18834666a071 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:38.275548 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:38.289377 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 972/1944] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:18:38.298399 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f016e65-ca5e-4ea7-b77f-3672c980950e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:38.299390 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f016e65-ca5e-4ea7-b77f-3672c980950e service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:38.299904 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f016e65-ca5e-4ea7-b77f-3672c980950e service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:38.300428 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f016e65-ca5e-4ea7-b77f-3672c980950e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:38.301154 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc0fc0-44ce-4a98-a8a3-411029aabcaf None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:38.302918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8f016e65-ca5e-4ea7-b77f-3672c980950e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:38.303237 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4ddc0fc0-44ce-4a98-a8a3-411029aabcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:38.315891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f016e65-ca5e-4ea7-b77f-3672c980950e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:38.317349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f016e65-ca5e-4ea7-b77f-3672c980950e service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:38.319437 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddc0fc0-44ce-4a98-a8a3-411029aabcaf None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:38.324562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324c492-f9f9-4428-8094-ededa8b9ee3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:38.325154 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324c492-f9f9-4428-8094-ededa8b9ee3d service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:38.325444 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324c492-f9f9-4428-8094-ededa8b9ee3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:38.325714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324c492-f9f9-4428-8094-ededa8b9ee3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:38.327977 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 973/1945] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:18:38.337947 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324c492-f9f9-4428-8094-ededa8b9ee3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:38.340827 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324c492-f9f9-4428-8094-ededa8b9ee3d service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:38.352433 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 973/1946] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:38.864567 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f016e65-ca5e-4ea7-b77f-3672c980950e service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:38.864567 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8f016e65-ca5e-4ea7-b77f-3672c980950e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:38.867051 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324c492-f9f9-4428-8094-ededa8b9ee3d service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:38.868464 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b324c492-f9f9-4428-8094-ededa8b9ee3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:38.882064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f016e65-ca5e-4ea7-b77f-3672c980950e service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:38.882214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324c492-f9f9-4428-8094-ededa8b9ee3d service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:38.894948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f0ae6c-9973-4704-8b79-301cfe393fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:38.898441 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ebd1e-f16b-48dd-9fca-d3be3780aa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:38.898922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0ae6c-9973-4704-8b79-301cfe393fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:38.899609 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0ae6c-9973-4704-8b79-301cfe393fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:38.899912 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f0ae6c-9973-4704-8b79-301cfe393fb0 admin admin] PATH_INFO: `/v3/projects/ce8010570e7f40bbafaf15b9330456c2` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:38.900698 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ebd1e-f16b-48dd-9fca-d3be3780aa43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:38.901177 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ebd1e-f16b-48dd-9fca-d3be3780aa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:38.901660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ebd1e-f16b-48dd-9fca-d3be3780aa43 admin admin] PATH_INFO: `/v3/projects/5c13a5c860de44a4950a165e8273fad0` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:38.901831 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f0ae6c-9973-4704-8b79-301cfe393fb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce8010570e7f40bbafaf15b9330456c2)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:38.905113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ebd1e-f16b-48dd-9fca-d3be3780aa43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c13a5c860de44a4950a165e8273fad0)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:38.906525 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f0ae6c-9973-4704-8b79-301cfe393fb0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:38.912185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ebd1e-f16b-48dd-9fca-d3be3780aa43 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:39.056080 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:39.069056 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-08f0ae6c-9973-4704-8b79-301cfe393fb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce8010570e7f40bbafaf15b9330456c2', 'request_id': 'req-08f0ae6c-9973-4704-8b79-301cfe393fb0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:39.072073 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bd1ebd1e-f16b-48dd-9fca-d3be3780aa43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c13a5c860de44a4950a165e8273fad0', 'request_id': 'req-bd1ebd1e-f16b-48dd-9fca-d3be3780aa43'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:39.077306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-08f0ae6c-9973-4704-8b79-301cfe393fb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce8010570e7f40bbafaf15b9330456c2', 'request_id': 'req-08f0ae6c-9973-4704-8b79-301cfe393fb0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:39.083723 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-08f0ae6c-9973-4704-8b79-301cfe393fb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce8010570e7f40bbafaf15b9330456c2', 'request_id': 'req-08f0ae6c-9973-4704-8b79-301cfe393fb0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:39.153663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bd1ebd1e-f16b-48dd-9fca-d3be3780aa43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c13a5c860de44a4950a165e8273fad0', 'request_id': 'req-bd1ebd1e-f16b-48dd-9fca-d3be3780aa43'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:39.160466 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bd1ebd1e-f16b-48dd-9fca-d3be3780aa43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c13a5c860de44a4950a165e8273fad0', 'request_id': 'req-bd1ebd1e-f16b-48dd-9fca-d3be3780aa43'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:39.174638 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bd1ebd1e-f16b-48dd-9fca-d3be3780aa43 admin admin] The token cache is being invalidate because project 5c13a5c860de44a4950a165e8273fad0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:39.174937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bd1ebd1e-f16b-48dd-9fca-d3be3780aa43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:39.176319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bd1ebd1e-f16b-48dd-9fca-d3be3780aa43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:39.178139 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 974/1947] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:38 2022] DELETE /identity/v3/projects/5c13a5c860de44a4950a165e8273fad0 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:18:39.182109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ebd1e-f16b-48dd-9fca-d3be3780aa43 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:39.183537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bd1ebd1e-f16b-48dd-9fca-d3be3780aa43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:39.218413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-08f0ae6c-9973-4704-8b79-301cfe393fb0 admin admin] The token cache is being invalidate because project ce8010570e7f40bbafaf15b9330456c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:39.219389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-08f0ae6c-9973-4704-8b79-301cfe393fb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:39.221987 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-08f0ae6c-9973-4704-8b79-301cfe393fb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:39.226010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 974/1948] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:38 2022] DELETE /identity/v3/projects/ce8010570e7f40bbafaf15b9330456c2 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:39.233870 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a070c1-2405-4850-bc65-538250bd982e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:39.234175 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a070c1-2405-4850-bc65-538250bd982e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:39.235641 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a070c1-2405-4850-bc65-538250bd982e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:39.279700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-19a070c1-2405-4850-bc65-538250bd982e None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:39.280943 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-19a070c1-2405-4850-bc65-538250bd982e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:39.310560 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ebd1e-f16b-48dd-9fca-d3be3780aa43 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:39.313149 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef410c7-b4fd-423e-808a-ca31e7af4392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:39.314112 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef410c7-b4fd-423e-808a-ca31e7af4392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:39.314590 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef410c7-b4fd-423e-808a-ca31e7af4392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:39.314876 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef410c7-b4fd-423e-808a-ca31e7af4392 admin admin] PATH_INFO: `/v3/users/a284b408e06c4737bc0098afe08f74e5` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:39.317209 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef410c7-b4fd-423e-808a-ca31e7af4392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a284b408e06c4737bc0098afe08f74e5)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:39.320587 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef410c7-b4fd-423e-808a-ca31e7af4392 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:39.349341 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 975/1949] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:39.358191 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57cd5e-7b5d-43f3-8242-94d9a37c2984 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:39.358532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57cd5e-7b5d-43f3-8242-94d9a37c2984 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:39.358824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57cd5e-7b5d-43f3-8242-94d9a37c2984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:39.378433 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5d57cd5e-7b5d-43f3-8242-94d9a37c2984 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:39.379965 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5d57cd5e-7b5d-43f3-8242-94d9a37c2984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:39.394631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 976/1950] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:18:39.395755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57cd5e-7b5d-43f3-8242-94d9a37c2984 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:39.406010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57cd5e-7b5d-43f3-8242-94d9a37c2984 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:39.410571 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4c228-77f8-47e4-8be9-0b4bbe78eee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:39.411221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4c228-77f8-47e4-8be9-0b4bbe78eee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:39.411547 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4c228-77f8-47e4-8be9-0b4bbe78eee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:39.411824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4c228-77f8-47e4-8be9-0b4bbe78eee0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:39.412549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4c228-77f8-47e4-8be9-0b4bbe78eee0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:18:39.412856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4c228-77f8-47e4-8be9-0b4bbe78eee0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:39.415712 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c4c228-77f8-47e4-8be9-0b4bbe78eee0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:39.430571 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 977/1951] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:18:39.431865 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4c228-77f8-47e4-8be9-0b4bbe78eee0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:39.442352 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4c228-77f8-47e4-8be9-0b4bbe78eee0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:39.447191 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61584e6e-238f-4caa-b18d-41145256f718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:39.447848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61584e6e-238f-4caa-b18d-41145256f718 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:39.448185 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61584e6e-238f-4caa-b18d-41145256f718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:39.448655 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61584e6e-238f-4caa-b18d-41145256f718 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:39.449159 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61584e6e-238f-4caa-b18d-41145256f718 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:39.452527 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61584e6e-238f-4caa-b18d-41145256f718 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:39.520847 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bef410c7-b4fd-423e-808a-ca31e7af4392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a284b408e06c4737bc0098afe08f74e5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:39.533430 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bef410c7-b4fd-423e-808a-ca31e7af4392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a284b408e06c4737bc0098afe08f74e5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:39.545697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 978/1952] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:18:39 2022] POST /identity/v3/projects => generated 397 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:18:39.549558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61584e6e-238f-4caa-b18d-41145256f718 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:39.575464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:39.612008 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61584e6e-238f-4caa-b18d-41145256f718 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:39.616759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6bd93-4847-4d60-a035-5605bc9295da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:39.617430 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6bd93-4847-4d60-a035-5605bc9295da admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:39.617718 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6bd93-4847-4d60-a035-5605bc9295da admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:39.617998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a6bd93-4847-4d60-a035-5605bc9295da admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:39.618733 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6bd93-4847-4d60-a035-5605bc9295da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:39.620670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a6bd93-4847-4d60-a035-5605bc9295da admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:39.645703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bef410c7-b4fd-423e-808a-ca31e7af4392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a284b408e06c4737bc0098afe08f74e5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:39.647032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bef410c7-b4fd-423e-808a-ca31e7af4392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a284b408e06c4737bc0098afe08f74e5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:39.648327 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bef410c7-b4fd-423e-808a-ca31e7af4392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a284b408e06c4737bc0098afe08f74e5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:39.668311 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 975/1953] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:39 2022] DELETE /identity/v3/users/a284b408e06c4737bc0098afe08f74e5 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:39.733279 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 979/1954] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:18:39 2022] POST /identity/v3/users => generated 299 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:18:39.737551 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef410c7-b4fd-423e-808a-ca31e7af4392 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:39.739026 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bef410c7-b4fd-423e-808a-ca31e7af4392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:39.830535 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef410c7-b4fd-423e-808a-ca31e7af4392 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:39.835346 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9048f-72f8-4ac0-8f53-2f11cfe1e81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:39.836041 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9048f-72f8-4ac0-8f53-2f11cfe1e81a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:39.836328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9048f-72f8-4ac0-8f53-2f11cfe1e81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:39.836610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9048f-72f8-4ac0-8f53-2f11cfe1e81a admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:39.837344 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9048f-72f8-4ac0-8f53-2f11cfe1e81a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:39.839111 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9048f-72f8-4ac0-8f53-2f11cfe1e81a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:39.855566 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 976/1955] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:39.855665 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6bd93-4847-4d60-a035-5605bc9295da admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:39.866805 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a6bd93-4847-4d60-a035-5605bc9295da admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:39.871623 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0513ee4-d9d0-43d2-9e01-f179d0d6847c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:39.872308 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0513ee4-d9d0-43d2-9e01-f179d0d6847c admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:39.873412 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0513ee4-d9d0-43d2-9e01-f179d0d6847c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:39.874013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0513ee4-d9d0-43d2-9e01-f179d0d6847c admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:39.874242 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0513ee4-d9d0-43d2-9e01-f179d0d6847c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:39.875901 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0513ee4-d9d0-43d2-9e01-f179d0d6847c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:39.890223 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 980/1956] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:18:39.892730 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9048f-72f8-4ac0-8f53-2f11cfe1e81a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:39.903219 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9048f-72f8-4ac0-8f53-2f11cfe1e81a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:39.908605 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78075021-1d69-4e87-8eaa-187851abb0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:39.909574 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78075021-1d69-4e87-8eaa-187851abb0a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:39.909950 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78075021-1d69-4e87-8eaa-187851abb0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:39.910288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78075021-1d69-4e87-8eaa-187851abb0a4 admin admin] PATH_INFO: `/v3/projects/79f25266fc974f9c93b2fcc97e8aad9f/users/3546f642fae74a7d806b2eae32f4aa96/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:39.930458 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78075021-1d69-4e87-8eaa-187851abb0a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79f25266fc974f9c93b2fcc97e8aad9f, user_id=3546f642fae74a7d806b2eae32f4aa96, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:39.933482 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78075021-1d69-4e87-8eaa-187851abb0a4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:40.014576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 977/1957] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:39 2022] PUT /identity/v3/projects/79f25266fc974f9c93b2fcc97e8aad9f/users/3546f642fae74a7d806b2eae32f4aa96/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:40.025492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702400ad-298e-4f5b-8c33-dedb1070ff59 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:40.025780 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702400ad-298e-4f5b-8c33-dedb1070ff59 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:40.026067 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702400ad-298e-4f5b-8c33-dedb1070ff59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:40.054117 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-702400ad-298e-4f5b-8c33-dedb1070ff59 None None] MFA Rules not processed for user `3546f642fae74a7d806b2eae32f4aa96`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:40.055373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-702400ad-298e-4f5b-8c33-dedb1070ff59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:40.119944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 981/1958] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:40.131665 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78075021-1d69-4e87-8eaa-187851abb0a4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:40.133713 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-78075021-1d69-4e87-8eaa-187851abb0a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:40.135498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702400ad-298e-4f5b-8c33-dedb1070ff59 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:40.136986 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-702400ad-298e-4f5b-8c33-dedb1070ff59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:40.199911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78075021-1d69-4e87-8eaa-187851abb0a4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:40.203864 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7868d7e6-f38e-49da-b4d2-3b33709269a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:40.204520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868d7e6-f38e-49da-b4d2-3b33709269a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:40.204807 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868d7e6-f38e-49da-b4d2-3b33709269a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:40.205122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7868d7e6-f38e-49da-b4d2-3b33709269a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:40.208680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702400ad-298e-4f5b-8c33-dedb1070ff59 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:40.213708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fa3c1-6458-4f3d-a578-22fa92b3aa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:40.214990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fa3c1-6458-4f3d-a578-22fa92b3aa67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:40.215099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868d7e6-f38e-49da-b4d2-3b33709269a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:40.215206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fa3c1-6458-4f3d-a578-22fa92b3aa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:40.215206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fa3c1-6458-4f3d-a578-22fa92b3aa67 admin admin] PATH_INFO: `/v3/projects/e741564bb69147b6b865ba5c5c5465f0` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:40.216868 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37fa3c1-6458-4f3d-a578-22fa92b3aa67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e741564bb69147b6b865ba5c5c5465f0)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:40.217057 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7868d7e6-f38e-49da-b4d2-3b33709269a8 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:40.219556 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37fa3c1-6458-4f3d-a578-22fa92b3aa67 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:40.229525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 978/1959] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:40.249466 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:40.405956 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d37fa3c1-6458-4f3d-a578-22fa92b3aa67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e741564bb69147b6b865ba5c5c5465f0', 'request_id': 'req-d37fa3c1-6458-4f3d-a578-22fa92b3aa67'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:40.407066 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d37fa3c1-6458-4f3d-a578-22fa92b3aa67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e741564bb69147b6b865ba5c5c5465f0', 'request_id': 'req-d37fa3c1-6458-4f3d-a578-22fa92b3aa67'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:40.414586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d37fa3c1-6458-4f3d-a578-22fa92b3aa67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e741564bb69147b6b865ba5c5c5465f0', 'request_id': 'req-d37fa3c1-6458-4f3d-a578-22fa92b3aa67'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:40.501252 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d37fa3c1-6458-4f3d-a578-22fa92b3aa67 admin admin] The token cache is being invalidate because project e741564bb69147b6b865ba5c5c5465f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:40.501998 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d37fa3c1-6458-4f3d-a578-22fa92b3aa67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:40.502959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d37fa3c1-6458-4f3d-a578-22fa92b3aa67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:40.505918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 982/1960] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:40 2022] DELETE /identity/v3/projects/e741564bb69147b6b865ba5c5c5465f0 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:40.511104 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203eb0f2-c520-4132-b0ef-fa712f13f156 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:40.511394 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203eb0f2-c520-4132-b0ef-fa712f13f156 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:40.511677 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203eb0f2-c520-4132-b0ef-fa712f13f156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:40.552011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-203eb0f2-c520-4132-b0ef-fa712f13f156 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:40.553858 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-203eb0f2-c520-4132-b0ef-fa712f13f156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:40.643161 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 979/1961] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:18:40.651261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc437a7c-1609-499f-8c0a-8b2dfa584e6a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:40.651550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc437a7c-1609-499f-8c0a-8b2dfa584e6a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:40.651832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc437a7c-1609-499f-8c0a-8b2dfa584e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:40.673511 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-fc437a7c-1609-499f-8c0a-8b2dfa584e6a None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:40.674756 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fc437a7c-1609-499f-8c0a-8b2dfa584e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:40.682660 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 983/1962] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:40.688092 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203eb0f2-c520-4132-b0ef-fa712f13f156 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:40.697811 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203eb0f2-c520-4132-b0ef-fa712f13f156 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:40.704384 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc509a-6589-4721-b45e-b1a6dfd522d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:40.705114 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc509a-6589-4721-b45e-b1a6dfd522d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:40.705331 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc509a-6589-4721-b45e-b1a6dfd522d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:40.705613 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc509a-6589-4721-b45e-b1a6dfd522d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:40.706446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc509a-6589-4721-b45e-b1a6dfd522d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:18:40.706759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc509a-6589-4721-b45e-b1a6dfd522d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:40.708375 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc509a-6589-4721-b45e-b1a6dfd522d3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:40.721684 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 980/1963] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:18:40.725843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc437a7c-1609-499f-8c0a-8b2dfa584e6a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:40.736001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc437a7c-1609-499f-8c0a-8b2dfa584e6a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:40.741440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6d0a5-0f46-4f93-9af5-f170257cdacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:40.742132 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6d0a5-0f46-4f93-9af5-f170257cdacc admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:40.742453 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6d0a5-0f46-4f93-9af5-f170257cdacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:40.742735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6d0a5-0f46-4f93-9af5-f170257cdacc admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:40.743435 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6d0a5-0f46-4f93-9af5-f170257cdacc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:40.746014 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6d0a5-0f46-4f93-9af5-f170257cdacc admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:40.828259 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:40.876210 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 984/1964] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:18:40 2022] POST /identity/v3/projects => generated 407 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:40.881109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc509a-6589-4721-b45e-b1a6dfd522d3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:40.892186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc509a-6589-4721-b45e-b1a6dfd522d3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:40.956850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f7921-a3a3-4fb7-82ca-1bad60a6c037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:40.957542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f7921-a3a3-4fb7-82ca-1bad60a6c037 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:40.957821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f7921-a3a3-4fb7-82ca-1bad60a6c037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:40.958104 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003f7921-a3a3-4fb7-82ca-1bad60a6c037 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:40.958916 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f7921-a3a3-4fb7-82ca-1bad60a6c037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:40.960867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003f7921-a3a3-4fb7-82ca-1bad60a6c037 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:41.066514 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 981/1965] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:18:40 2022] POST /identity/v3/users => generated 304 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:41.074906 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6d0a5-0f46-4f93-9af5-f170257cdacc admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:41.081464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6d0a5-0f46-4f93-9af5-f170257cdacc admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:41.085917 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37711fb1-1a38-48d2-8da7-d4c228fde3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:41.086749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37711fb1-1a38-48d2-8da7-d4c228fde3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:41.087037 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37711fb1-1a38-48d2-8da7-d4c228fde3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:41.087314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37711fb1-1a38-48d2-8da7-d4c228fde3a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:41.088048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37711fb1-1a38-48d2-8da7-d4c228fde3a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:41.089650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37711fb1-1a38-48d2-8da7-d4c228fde3a1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:41.101011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 985/1966] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:41.106632 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f7921-a3a3-4fb7-82ca-1bad60a6c037 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:41.128901 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003f7921-a3a3-4fb7-82ca-1bad60a6c037 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:41.134140 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38732bc-85b4-4b08-bb48-73a884f20cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:41.134815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38732bc-85b4-4b08-bb48-73a884f20cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:41.135105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38732bc-85b4-4b08-bb48-73a884f20cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:41.135391 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38732bc-85b4-4b08-bb48-73a884f20cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:41.136130 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38732bc-85b4-4b08-bb48-73a884f20cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:41.137782 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38732bc-85b4-4b08-bb48-73a884f20cf5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:41.156179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37711fb1-1a38-48d2-8da7-d4c228fde3a1 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:41.156513 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:41.157317 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 982/1967] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:41.171040 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37711fb1-1a38-48d2-8da7-d4c228fde3a1 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:41.175824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc0957-ebdb-4835-b43e-70f033cf3e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:41.177206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc0957-ebdb-4835-b43e-70f033cf3e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:41.177206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc0957-ebdb-4835-b43e-70f033cf3e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:41.177491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc0957-ebdb-4835-b43e-70f033cf3e18 admin admin] PATH_INFO: `/v3/projects/8724cc790d8d4c01b2ab642574728dcd/users/1ff41b6fdf254d1599564870b2073940/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:41.193184 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc0957-ebdb-4835-b43e-70f033cf3e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8724cc790d8d4c01b2ab642574728dcd, user_id=1ff41b6fdf254d1599564870b2073940, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:41.196213 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc0957-ebdb-4835-b43e-70f033cf3e18 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:41.290995 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 986/1968] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:41 2022] PUT /identity/v3/projects/8724cc790d8d4c01b2ab642574728dcd/users/1ff41b6fdf254d1599564870b2073940/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:18:41.302924 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d9318-52c8-4fe1-a3db-de00fcaf28c9 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:41.303218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d9318-52c8-4fe1-a3db-de00fcaf28c9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:41.303511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26d9318-52c8-4fe1-a3db-de00fcaf28c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:41.332632 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f26d9318-52c8-4fe1-a3db-de00fcaf28c9 None None] MFA Rules not processed for user `1ff41b6fdf254d1599564870b2073940`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:41.333863 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f26d9318-52c8-4fe1-a3db-de00fcaf28c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:41.407636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 983/1969] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:41 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:41.412125 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc0957-ebdb-4835-b43e-70f033cf3e18 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:41.474195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc0957-ebdb-4835-b43e-70f033cf3e18 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:41.479688 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005d335-b3ee-449f-b077-10a19fa8f6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:41.480477 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005d335-b3ee-449f-b077-10a19fa8f6b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:41.480861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005d335-b3ee-449f-b077-10a19fa8f6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:41.481242 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005d335-b3ee-449f-b077-10a19fa8f6b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:41.482043 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005d335-b3ee-449f-b077-10a19fa8f6b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:41.484742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005d335-b3ee-449f-b077-10a19fa8f6b5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:41.576875 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 987/1970] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:18:41 2022] POST /identity/v3/projects => generated 405 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:41.581556 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d9318-52c8-4fe1-a3db-de00fcaf28c9 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:41.592997 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26d9318-52c8-4fe1-a3db-de00fcaf28c9 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:41.678884 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7d092-acdd-4d0c-82e7-3c10ea584df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:41.679656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7d092-acdd-4d0c-82e7-3c10ea584df5 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:41.681262 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7d092-acdd-4d0c-82e7-3c10ea584df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:41.681262 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7d092-acdd-4d0c-82e7-3c10ea584df5 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:41.681262 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7d092-acdd-4d0c-82e7-3c10ea584df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:41.683558 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7d092-acdd-4d0c-82e7-3c10ea584df5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:41.804033 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 984/1971] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:18:41 2022] POST /identity/v3/users => generated 309 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:41.819811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005d335-b3ee-449f-b077-10a19fa8f6b5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:41.837153 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005d335-b3ee-449f-b077-10a19fa8f6b5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:41.840805 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107653d-20cf-4073-9787-d465fac9370e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:41.842127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107653d-20cf-4073-9787-d465fac9370e admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:41.842626 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107653d-20cf-4073-9787-d465fac9370e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:41.842983 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6107653d-20cf-4073-9787-d465fac9370e admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:41.843834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107653d-20cf-4073-9787-d465fac9370e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:41.845576 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6107653d-20cf-4073-9787-d465fac9370e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:41.857032 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 988/1972] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:41.861449 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7d092-acdd-4d0c-82e7-3c10ea584df5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:41.871551 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7d092-acdd-4d0c-82e7-3c10ea584df5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:41.876951 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9df8a-eb6d-4389-b46e-3c8d4cc48b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:41.877823 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9df8a-eb6d-4389-b46e-3c8d4cc48b83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:41.878109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9df8a-eb6d-4389-b46e-3c8d4cc48b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:41.878402 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9df8a-eb6d-4389-b46e-3c8d4cc48b83 admin admin] PATH_INFO: `/v3/projects/2bd67c36893f40bfbd60177ddf2a2483/users/df856445e86a4edbb25689a2b5cea3b9/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:41.895317 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9df8a-eb6d-4389-b46e-3c8d4cc48b83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bd67c36893f40bfbd60177ddf2a2483, user_id=df856445e86a4edbb25689a2b5cea3b9, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:41.898281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9df8a-eb6d-4389-b46e-3c8d4cc48b83 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:42.017005 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 985/1973] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:41 2022] PUT /identity/v3/projects/2bd67c36893f40bfbd60177ddf2a2483/users/df856445e86a4edbb25689a2b5cea3b9/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:42.020831 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107653d-20cf-4073-9787-d465fac9370e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:42.031311 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6107653d-20cf-4073-9787-d465fac9370e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:42.061149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:42.088274 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3d8333-9222-4e54-8c5b-040f7e8ffea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:42.089226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d8333-9222-4e54-8c5b-040f7e8ffea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:42.089517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d8333-9222-4e54-8c5b-040f7e8ffea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:42.089802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3d8333-9222-4e54-8c5b-040f7e8ffea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:42.090565 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d8333-9222-4e54-8c5b-040f7e8ffea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:42.092195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3d8333-9222-4e54-8c5b-040f7e8ffea1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:42.101885 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 989/1974] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:42.106222 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9df8a-eb6d-4389-b46e-3c8d4cc48b83 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:42.117159 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9df8a-eb6d-4389-b46e-3c8d4cc48b83 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:42.121724 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e546c6a-059d-4128-bef4-1e342a623e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:42.122646 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e546c6a-059d-4128-bef4-1e342a623e51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:42.122935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e546c6a-059d-4128-bef4-1e342a623e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:42.123262 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e546c6a-059d-4128-bef4-1e342a623e51 admin admin] PATH_INFO: `/v3/projects/2bd67c36893f40bfbd60177ddf2a2483/users/df856445e86a4edbb25689a2b5cea3b9/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:42.125513 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e546c6a-059d-4128-bef4-1e342a623e51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bd67c36893f40bfbd60177ddf2a2483, user_id=df856445e86a4edbb25689a2b5cea3b9, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:42.128281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e546c6a-059d-4128-bef4-1e342a623e51 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:42.138513 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 986/1975] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:42 2022] PUT /identity/v3/projects/2bd67c36893f40bfbd60177ddf2a2483/users/df856445e86a4edbb25689a2b5cea3b9/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:42.150093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90f1e5-7759-42a7-b8f1-3ab1c0e89a17 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:42.150402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90f1e5-7759-42a7-b8f1-3ab1c0e89a17 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:42.150711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90f1e5-7759-42a7-b8f1-3ab1c0e89a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:42.176897 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-fe90f1e5-7759-42a7-b8f1-3ab1c0e89a17 None None] MFA Rules not processed for user `df856445e86a4edbb25689a2b5cea3b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:42.178088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fe90f1e5-7759-42a7-b8f1-3ab1c0e89a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:42.263170 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 990/1976] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:42 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:42.274481 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e546c6a-059d-4128-bef4-1e342a623e51 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:42.276175 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8e546c6a-059d-4128-bef4-1e342a623e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:42.282050 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:42.333261 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e546c6a-059d-4128-bef4-1e342a623e51 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:42.336901 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33ed34-fd07-450f-a472-d6ad0f6779ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:42.337537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33ed34-fd07-450f-a472-d6ad0f6779ba service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:42.337819 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33ed34-fd07-450f-a472-d6ad0f6779ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:42.338096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33ed34-fd07-450f-a472-d6ad0f6779ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:42.347671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33ed34-fd07-450f-a472-d6ad0f6779ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:42.349084 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33ed34-fd07-450f-a472-d6ad0f6779ba service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:42.359075 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 987/1977] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:42 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:42.792745 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90f1e5-7759-42a7-b8f1-3ab1c0e89a17 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:42.805581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90f1e5-7759-42a7-b8f1-3ab1c0e89a17 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:42.809755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e7ae5-2a90-4182-8c1b-09273ef4cacb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:42.810441 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e7ae5-2a90-4182-8c1b-09273ef4cacb service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:42.818467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e7ae5-2a90-4182-8c1b-09273ef4cacb service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:42.818467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e7ae5-2a90-4182-8c1b-09273ef4cacb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:42.893585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e7ae5-2a90-4182-8c1b-09273ef4cacb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:42.895265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e7ae5-2a90-4182-8c1b-09273ef4cacb service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:42.906480 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 991/1978] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:42 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:51.556313 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33ed34-fd07-450f-a472-d6ad0f6779ba service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:51.559674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bf33ed34-fd07-450f-a472-d6ad0f6779ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:51.650810 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33ed34-fd07-450f-a472-d6ad0f6779ba service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:51.657210 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adee7a-6936-4d03-a6dc-3800147f9d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:51.658122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adee7a-6936-4d03-a6dc-3800147f9d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:51.658452 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adee7a-6936-4d03-a6dc-3800147f9d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:51.658790 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62adee7a-6936-4d03-a6dc-3800147f9d93 admin admin] PATH_INFO: `/v3/users/3546f642fae74a7d806b2eae32f4aa96` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:51.660323 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62adee7a-6936-4d03-a6dc-3800147f9d93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3546f642fae74a7d806b2eae32f4aa96)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:51.662828 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62adee7a-6936-4d03-a6dc-3800147f9d93 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:51.890865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-62adee7a-6936-4d03-a6dc-3800147f9d93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3546f642fae74a7d806b2eae32f4aa96'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:51.909806 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-62adee7a-6936-4d03-a6dc-3800147f9d93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3546f642fae74a7d806b2eae32f4aa96'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:52.037433 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-62adee7a-6936-4d03-a6dc-3800147f9d93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3546f642fae74a7d806b2eae32f4aa96'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:52.037433 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-62adee7a-6936-4d03-a6dc-3800147f9d93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3546f642fae74a7d806b2eae32f4aa96'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:52.037433 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-62adee7a-6936-4d03-a6dc-3800147f9d93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3546f642fae74a7d806b2eae32f4aa96'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:52.057610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 988/1979] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:51 2022] DELETE /identity/v3/users/3546f642fae74a7d806b2eae32f4aa96 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:52.062855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62846e92-1ac9-49ed-b45a-486a51e47efa None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:52.067774 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62846e92-1ac9-49ed-b45a-486a51e47efa None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:52.067774 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62846e92-1ac9-49ed-b45a-486a51e47efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:52.069037 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adee7a-6936-4d03-a6dc-3800147f9d93 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:52.070531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-62adee7a-6936-4d03-a6dc-3800147f9d93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:52.095989 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-62846e92-1ac9-49ed-b45a-486a51e47efa None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:52.095989 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-62846e92-1ac9-49ed-b45a-486a51e47efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:52.104850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:52.142419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:52.142900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62adee7a-6936-4d03-a6dc-3800147f9d93 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:52.147813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1cbd11-2701-4d6b-9db2-62b56e95d2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:52.148764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cbd11-2701-4d6b-9db2-62b56e95d2a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:52.149049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cbd11-2701-4d6b-9db2-62b56e95d2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:52.149321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1cbd11-2701-4d6b-9db2-62b56e95d2a4 admin admin] PATH_INFO: `/v3/users/aa8237a791034b80973d5cc4329d015c` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:52.150565 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1cbd11-2701-4d6b-9db2-62b56e95d2a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa8237a791034b80973d5cc4329d015c)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:52.152501 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1cbd11-2701-4d6b-9db2-62b56e95d2a4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:52.157409 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 992/1980] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:52.167241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62846e92-1ac9-49ed-b45a-486a51e47efa None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:52.168667 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-62846e92-1ac9-49ed-b45a-486a51e47efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:52.228975 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62846e92-1ac9-49ed-b45a-486a51e47efa None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:52.233347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dab389-6428-48b9-aefb-330364490734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:52.234002 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dab389-6428-48b9-aefb-330364490734 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:52.234350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dab389-6428-48b9-aefb-330364490734 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:52.234670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dab389-6428-48b9-aefb-330364490734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:52.244424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dab389-6428-48b9-aefb-330364490734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:52.246090 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dab389-6428-48b9-aefb-330364490734 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:52.259298 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 993/1981] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:18:52.332588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bd1cbd11-2701-4d6b-9db2-62b56e95d2a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa8237a791034b80973d5cc4329d015c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:52.349076 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bd1cbd11-2701-4d6b-9db2-62b56e95d2a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa8237a791034b80973d5cc4329d015c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:52.433648 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bd1cbd11-2701-4d6b-9db2-62b56e95d2a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa8237a791034b80973d5cc4329d015c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:52.434286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bd1cbd11-2701-4d6b-9db2-62b56e95d2a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa8237a791034b80973d5cc4329d015c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:52.434949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bd1cbd11-2701-4d6b-9db2-62b56e95d2a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa8237a791034b80973d5cc4329d015c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:52.449294 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 989/1982] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:52 2022] DELETE /identity/v3/users/aa8237a791034b80973d5cc4329d015c => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:18:52.835699 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dab389-6428-48b9-aefb-330364490734 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:52.842266 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1cbd11-2701-4d6b-9db2-62b56e95d2a4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:52.843980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bd1cbd11-2701-4d6b-9db2-62b56e95d2a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:52.846333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dab389-6428-48b9-aefb-330364490734 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:52.938944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ead23c9-b3f4-4231-ab1d-8f95f97c28ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:52.939942 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead23c9-b3f4-4231-ab1d-8f95f97c28ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:52.940170 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1cbd11-2701-4d6b-9db2-62b56e95d2a4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:52.940414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead23c9-b3f4-4231-ab1d-8f95f97c28ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:52.940792 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ead23c9-b3f4-4231-ab1d-8f95f97c28ac admin admin] PATH_INFO: `/v3/projects/a48b90744f124a21b9e8d2b78836462e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:52.942051 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead23c9-b3f4-4231-ab1d-8f95f97c28ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=a48b90744f124a21b9e8d2b78836462e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:52.944099 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ead23c9-b3f4-4231-ab1d-8f95f97c28ac admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:52.945159 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a8378-ab48-40cc-a013-cf9bbfb33df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:52.946082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a8378-ab48-40cc-a013-cf9bbfb33df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:52.946322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a8378-ab48-40cc-a013-cf9bbfb33df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:52.946677 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a8378-ab48-40cc-a013-cf9bbfb33df2 admin admin] PATH_INFO: `/v3/projects/79f25266fc974f9c93b2fcc97e8aad9f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:52.947932 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a8378-ab48-40cc-a013-cf9bbfb33df2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79f25266fc974f9c93b2fcc97e8aad9f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:52.949993 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a8378-ab48-40cc-a013-cf9bbfb33df2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:53.102033 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5ead23c9-b3f4-4231-ab1d-8f95f97c28ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a48b90744f124a21b9e8d2b78836462e', 'request_id': 'req-5ead23c9-b3f4-4231-ab1d-8f95f97c28ac'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:53.110602 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5ead23c9-b3f4-4231-ab1d-8f95f97c28ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a48b90744f124a21b9e8d2b78836462e', 'request_id': 'req-5ead23c9-b3f4-4231-ab1d-8f95f97c28ac'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:53.112117 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:53.116331 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8f9a8378-ab48-40cc-a013-cf9bbfb33df2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79f25266fc974f9c93b2fcc97e8aad9f', 'request_id': 'req-8f9a8378-ab48-40cc-a013-cf9bbfb33df2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:53.118683 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5ead23c9-b3f4-4231-ab1d-8f95f97c28ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a48b90744f124a21b9e8d2b78836462e', 'request_id': 'req-5ead23c9-b3f4-4231-ab1d-8f95f97c28ac'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:53.187506 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8f9a8378-ab48-40cc-a013-cf9bbfb33df2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79f25266fc974f9c93b2fcc97e8aad9f', 'request_id': 'req-8f9a8378-ab48-40cc-a013-cf9bbfb33df2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:53.193869 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8f9a8378-ab48-40cc-a013-cf9bbfb33df2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79f25266fc974f9c93b2fcc97e8aad9f', 'request_id': 'req-8f9a8378-ab48-40cc-a013-cf9bbfb33df2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:53.209633 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8f9a8378-ab48-40cc-a013-cf9bbfb33df2 admin admin] The token cache is being invalidate because project 79f25266fc974f9c93b2fcc97e8aad9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:53.209937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8f9a8378-ab48-40cc-a013-cf9bbfb33df2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:53.210661 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8f9a8378-ab48-40cc-a013-cf9bbfb33df2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:53.212578 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 990/1983] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:52 2022] DELETE /identity/v3/projects/79f25266fc974f9c93b2fcc97e8aad9f => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:53.218992 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9c25f-9512-48f0-9c3e-6d62180fd3c0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:53.219341 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9c25f-9512-48f0-9c3e-6d62180fd3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:53.219799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9c25f-9512-48f0-9c3e-6d62180fd3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:53.240729 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2cf9c25f-9512-48f0-9c3e-6d62180fd3c0 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:53.242505 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2cf9c25f-9512-48f0-9c3e-6d62180fd3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:53.251289 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5ead23c9-b3f4-4231-ab1d-8f95f97c28ac admin admin] The token cache is being invalidate because project a48b90744f124a21b9e8d2b78836462e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:53.251589 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5ead23c9-b3f4-4231-ab1d-8f95f97c28ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:53.254171 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5ead23c9-b3f4-4231-ab1d-8f95f97c28ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:53.255477 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 994/1984] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:52 2022] DELETE /identity/v3/projects/a48b90744f124a21b9e8d2b78836462e => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:53.260653 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcf216-b0a5-4be6-8705-9b591a9a3040 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:53.260944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcf216-b0a5-4be6-8705-9b591a9a3040 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:53.261232 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcf216-b0a5-4be6-8705-9b591a9a3040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:53.302426 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-fbdcf216-b0a5-4be6-8705-9b591a9a3040 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:53.303783 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fbdcf216-b0a5-4be6-8705-9b591a9a3040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:53.315385 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 991/1985] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:18:53.326474 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:53.326753 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f5bd6-e0df-4b38-a489-8aa3b0c43a64 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:53.327357 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f5bd6-e0df-4b38-a489-8aa3b0c43a64 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:53.327890 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6f5bd6-e0df-4b38-a489-8aa3b0c43a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:53.347220 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9d6f5bd6-e0df-4b38-a489-8aa3b0c43a64 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:53.348445 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9d6f5bd6-e0df-4b38-a489-8aa3b0c43a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:53.357944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 992/1986] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:18:53.362004 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f5bd6-e0df-4b38-a489-8aa3b0c43a64 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:53.371115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6f5bd6-e0df-4b38-a489-8aa3b0c43a64 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:53.371735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 995/1987] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:53.377104 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14581cc-6fc9-474e-bb28-656d0c93c35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:53.377766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14581cc-6fc9-474e-bb28-656d0c93c35b admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:53.378053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14581cc-6fc9-474e-bb28-656d0c93c35b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:53.378329 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14581cc-6fc9-474e-bb28-656d0c93c35b admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:53.379244 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14581cc-6fc9-474e-bb28-656d0c93c35b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:18:53.379558 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14581cc-6fc9-474e-bb28-656d0c93c35b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:53.380678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d460346-34cd-4ef3-a662-94df13a9eb49 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:53.380977 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d460346-34cd-4ef3-a662-94df13a9eb49 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:53.381228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14581cc-6fc9-474e-bb28-656d0c93c35b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:53.381343 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d460346-34cd-4ef3-a662-94df13a9eb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:53.394801 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 993/1988] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:18:53.400214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14581cc-6fc9-474e-bb28-656d0c93c35b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:53.403701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6d460346-34cd-4ef3-a662-94df13a9eb49 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:53.404950 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6d460346-34cd-4ef3-a662-94df13a9eb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:53.413514 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14581cc-6fc9-474e-bb28-656d0c93c35b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:53.413659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 996/1989] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:53.416188 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38de512-92d0-40bb-8f49-f49230d9f5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:53.417569 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38de512-92d0-40bb-8f49-f49230d9f5e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:53.417698 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d460346-34cd-4ef3-a662-94df13a9eb49 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:53.417975 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38de512-92d0-40bb-8f49-f49230d9f5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:53.418262 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38de512-92d0-40bb-8f49-f49230d9f5e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:53.419024 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38de512-92d0-40bb-8f49-f49230d9f5e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:53.421758 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38de512-92d0-40bb-8f49-f49230d9f5e1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:53.428614 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d460346-34cd-4ef3-a662-94df13a9eb49 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:53.433336 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e4405-effc-4b50-86fa-ffc0b7ad6c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:53.434002 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e4405-effc-4b50-86fa-ffc0b7ad6c6a admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:53.434286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e4405-effc-4b50-86fa-ffc0b7ad6c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:53.434620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e4405-effc-4b50-86fa-ffc0b7ad6c6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:53.436661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e4405-effc-4b50-86fa-ffc0b7ad6c6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:18:53.442110 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e4405-effc-4b50-86fa-ffc0b7ad6c6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:53.442110 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e4405-effc-4b50-86fa-ffc0b7ad6c6a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:53.450584 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 997/1990] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:18:53.458410 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e4405-effc-4b50-86fa-ffc0b7ad6c6a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:53.469571 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e4405-effc-4b50-86fa-ffc0b7ad6c6a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:53.474464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b5c8c-a0bc-47f3-b53a-6167ae59b4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:53.475120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b5c8c-a0bc-47f3-b53a-6167ae59b4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:53.475402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b5c8c-a0bc-47f3-b53a-6167ae59b4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:53.475800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b5c8c-a0bc-47f3-b53a-6167ae59b4f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:53.476494 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b5c8c-a0bc-47f3-b53a-6167ae59b4f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:53.478433 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b5c8c-a0bc-47f3-b53a-6167ae59b4f6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:53.533340 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 994/1991] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:18:53 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:53.545065 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38de512-92d0-40bb-8f49-f49230d9f5e1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:53.599945 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38de512-92d0-40bb-8f49-f49230d9f5e1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:53.604651 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7acb5-3829-4bcd-8570-c5e2da620070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:53.605318 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7acb5-3829-4bcd-8570-c5e2da620070 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:53.605605 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7acb5-3829-4bcd-8570-c5e2da620070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:53.605891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7acb5-3829-4bcd-8570-c5e2da620070 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:53.606614 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7acb5-3829-4bcd-8570-c5e2da620070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:53.608543 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7acb5-3829-4bcd-8570-c5e2da620070 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:53.615764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 998/1992] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:18:53 2022] POST /identity/v3/projects => generated 369 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:18:53.617086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b5c8c-a0bc-47f3-b53a-6167ae59b4f6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:53.683136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b5c8c-a0bc-47f3-b53a-6167ae59b4f6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:53.689827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c2670-ee12-4fe9-a3c5-b40e8d239cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:53.690541 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c2670-ee12-4fe9-a3c5-b40e8d239cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:53.690828 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c2670-ee12-4fe9-a3c5-b40e8d239cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:53.691106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c2670-ee12-4fe9-a3c5-b40e8d239cf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:53.691802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c2670-ee12-4fe9-a3c5-b40e8d239cf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:53.693966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8c2670-ee12-4fe9-a3c5-b40e8d239cf5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:53.722633 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 995/1993] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:18:53 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:18:53.723964 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7acb5-3829-4bcd-8570-c5e2da620070 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:53.735892 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7acb5-3829-4bcd-8570-c5e2da620070 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:53.740605 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b618dce-c837-4d2e-8bad-c9fe40810c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:53.741273 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b618dce-c837-4d2e-8bad-c9fe40810c4f admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:53.741556 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b618dce-c837-4d2e-8bad-c9fe40810c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:53.741835 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b618dce-c837-4d2e-8bad-c9fe40810c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:53.742740 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b618dce-c837-4d2e-8bad-c9fe40810c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:53.744780 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b618dce-c837-4d2e-8bad-c9fe40810c4f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:53.756687 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 996/1994] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:18:53.760767 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b618dce-c837-4d2e-8bad-c9fe40810c4f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:53.773812 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b618dce-c837-4d2e-8bad-c9fe40810c4f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:53.779171 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26f630-9f97-4567-9d36-62d38474685d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:53.779838 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26f630-9f97-4567-9d36-62d38474685d admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:53.780120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26f630-9f97-4567-9d36-62d38474685d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:53.780397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26f630-9f97-4567-9d36-62d38474685d admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:53.781352 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac26f630-9f97-4567-9d36-62d38474685d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:53.782774 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac26f630-9f97-4567-9d36-62d38474685d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:53.788364 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:53.794112 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 997/1995] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:18:53.795851 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 999/1996] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:18:53 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:18:53.799505 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c2670-ee12-4fe9-a3c5-b40e8d239cf5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:53.799644 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26f630-9f97-4567-9d36-62d38474685d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:53.812181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c2670-ee12-4fe9-a3c5-b40e8d239cf5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:53.812783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26f630-9f97-4567-9d36-62d38474685d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:53.817977 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16805f5c-2b8d-4c49-a9f5-ce0546aeab20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:53.818536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad3c8d-c760-49c0-9be1-18cd001bc689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:53.818918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16805f5c-2b8d-4c49-a9f5-ce0546aeab20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:53.819205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16805f5c-2b8d-4c49-a9f5-ce0546aeab20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:53.819309 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad3c8d-c760-49c0-9be1-18cd001bc689 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:53.819483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16805f5c-2b8d-4c49-a9f5-ce0546aeab20 admin admin] PATH_INFO: `/v3/projects/7e75e404ef4f4118a95ca8fb8d98becd/users/f3821e46c09b449d85c46c58d99bf607/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:53.819583 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad3c8d-c760-49c0-9be1-18cd001bc689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:53.819841 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad3c8d-c760-49c0-9be1-18cd001bc689 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:53.820786 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad3c8d-c760-49c0-9be1-18cd001bc689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:53.822440 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad3c8d-c760-49c0-9be1-18cd001bc689 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:53.837961 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 998/1997] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:53.838996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad3c8d-c760-49c0-9be1-18cd001bc689 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:53.852925 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16805f5c-2b8d-4c49-a9f5-ce0546aeab20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e75e404ef4f4118a95ca8fb8d98becd, user_id=f3821e46c09b449d85c46c58d99bf607, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:53.852925 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16805f5c-2b8d-4c49-a9f5-ce0546aeab20 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:53.854249 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad3c8d-c760-49c0-9be1-18cd001bc689 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:53.856597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63621052-8378-4efe-abfa-5dcde830ea1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:53.858014 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63621052-8378-4efe-abfa-5dcde830ea1d admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:53.858356 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63621052-8378-4efe-abfa-5dcde830ea1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:53.858677 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63621052-8378-4efe-abfa-5dcde830ea1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:53.859405 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63621052-8378-4efe-abfa-5dcde830ea1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:53.861363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63621052-8378-4efe-abfa-5dcde830ea1d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:53.871917 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 999/1998] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:53.878075 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63621052-8378-4efe-abfa-5dcde830ea1d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:53.888053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63621052-8378-4efe-abfa-5dcde830ea1d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:53.893653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062dd9b-7005-4129-82bb-73db076d4173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:53.894608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062dd9b-7005-4129-82bb-73db076d4173 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:53.894897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062dd9b-7005-4129-82bb-73db076d4173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:53.895186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062dd9b-7005-4129-82bb-73db076d4173 admin admin] PATH_INFO: `/v3/projects/0c2337d47cc14154bb1ec9d768f8aad9/users/ef679b01efe4428aa4dad0b3af8af5ac/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:53.911216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062dd9b-7005-4129-82bb-73db076d4173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c2337d47cc14154bb1ec9d768f8aad9, user_id=ef679b01efe4428aa4dad0b3af8af5ac, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:53.913926 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062dd9b-7005-4129-82bb-73db076d4173 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:53.959443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1000/1999] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:53 2022] PUT /identity/v3/projects/7e75e404ef4f4118a95ca8fb8d98becd/users/f3821e46c09b449d85c46c58d99bf607/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:53.964231 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89452a62-e36b-4300-b735-1cfef81be8c4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:53.964517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89452a62-e36b-4300-b735-1cfef81be8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:53.964791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89452a62-e36b-4300-b735-1cfef81be8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:53.981901 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-89452a62-e36b-4300-b735-1cfef81be8c4 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:53.983182 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-89452a62-e36b-4300-b735-1cfef81be8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:54.040755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1001/2000] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:18:54.050903 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89452a62-e36b-4300-b735-1cfef81be8c4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:54.052270 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-89452a62-e36b-4300-b735-1cfef81be8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:54.056745 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1000/2001] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:53 2022] PUT /identity/v3/projects/0c2337d47cc14154bb1ec9d768f8aad9/users/ef679b01efe4428aa4dad0b3af8af5ac/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:54.066910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d71c3-729c-498d-ab7b-0b542cd0b875 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:54.067359 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d71c3-729c-498d-ab7b-0b542cd0b875 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:54.067628 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378d71c3-729c-498d-ab7b-0b542cd0b875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:54.075342 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:54.096846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-378d71c3-729c-498d-ab7b-0b542cd0b875 None None] MFA Rules not processed for user `ef679b01efe4428aa4dad0b3af8af5ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:54.098122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-378d71c3-729c-498d-ab7b-0b542cd0b875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:54.114813 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89452a62-e36b-4300-b735-1cfef81be8c4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:54.166623 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1001/2002] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:54 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:18:54.170346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7df6b-1baa-4af0-8a0e-760590f322b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:54.171910 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7df6b-1baa-4af0-8a0e-760590f322b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:54.172058 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d71c3-729c-498d-ab7b-0b542cd0b875 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:54.172451 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7df6b-1baa-4af0-8a0e-760590f322b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:54.172987 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7df6b-1baa-4af0-8a0e-760590f322b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:54.248179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7df6b-1baa-4af0-8a0e-760590f322b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:54.249950 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b7df6b-1baa-4af0-8a0e-760590f322b5 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:54.260102 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1002/2003] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:18:54.264692 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378d71c3-729c-498d-ab7b-0b542cd0b875 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:54.269510 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c00a56-0fc2-4dec-9646-71ba6d69bd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:54.270204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c00a56-0fc2-4dec-9646-71ba6d69bd42 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:54.270557 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c00a56-0fc2-4dec-9646-71ba6d69bd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:54.270795 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c00a56-0fc2-4dec-9646-71ba6d69bd42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:54.271650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c00a56-0fc2-4dec-9646-71ba6d69bd42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:54.274137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c00a56-0fc2-4dec-9646-71ba6d69bd42 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:54.370327 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1002/2004] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:18:54 2022] POST /identity/v3/projects => generated 369 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:18:54.371824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7df6b-1baa-4af0-8a0e-760590f322b5 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:54.382174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7df6b-1baa-4af0-8a0e-760590f322b5 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:54.440548 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:54.447915 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6775cb8-6115-4f22-915d-26fe22d1d339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:54.448641 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6775cb8-6115-4f22-915d-26fe22d1d339 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:54.448898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6775cb8-6115-4f22-915d-26fe22d1d339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:54.449256 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6775cb8-6115-4f22-915d-26fe22d1d339 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:54.449884 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6775cb8-6115-4f22-915d-26fe22d1d339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:54.452019 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6775cb8-6115-4f22-915d-26fe22d1d339 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:54.571795 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1003/2005] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:18:54 2022] POST /identity/v3/users => generated 291 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:54.573324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c00a56-0fc2-4dec-9646-71ba6d69bd42 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:54.586473 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c00a56-0fc2-4dec-9646-71ba6d69bd42 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:54.591897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed10a04-c0a0-4afc-9f42-ca49fa30cfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:54.592592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed10a04-c0a0-4afc-9f42-ca49fa30cfb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:54.592881 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed10a04-c0a0-4afc-9f42-ca49fa30cfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:54.593158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed10a04-c0a0-4afc-9f42-ca49fa30cfb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:54.594032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed10a04-c0a0-4afc-9f42-ca49fa30cfb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:54.595796 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed10a04-c0a0-4afc-9f42-ca49fa30cfb8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:54.613226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1003/2006] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:54.614403 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6775cb8-6115-4f22-915d-26fe22d1d339 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:54.628479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6775cb8-6115-4f22-915d-26fe22d1d339 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:54.634055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e116e-b8df-48e8-9fbf-73e6692564ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:54.635069 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e116e-b8df-48e8-9fbf-73e6692564ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:54.635562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e116e-b8df-48e8-9fbf-73e6692564ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:54.636488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e116e-b8df-48e8-9fbf-73e6692564ef admin admin] PATH_INFO: `/v3/projects/4527ff8365ac407085e35fe07e530bba/users/fbce3eaa17c54629bee157bb7620c74c/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:54.658655 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e116e-b8df-48e8-9fbf-73e6692564ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=4527ff8365ac407085e35fe07e530bba, user_id=fbce3eaa17c54629bee157bb7620c74c, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:54.661955 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e116e-b8df-48e8-9fbf-73e6692564ef admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:54.812729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1004/2007] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:54 2022] PUT /identity/v3/projects/4527ff8365ac407085e35fe07e530bba/users/fbce3eaa17c54629bee157bb7620c74c/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:54.816676 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed10a04-c0a0-4afc-9f42-ca49fa30cfb8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:54.829110 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed10a04-c0a0-4afc-9f42-ca49fa30cfb8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:54.843027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:54.888987 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7eef9-b767-46fc-9363-1ddca168caec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:54.889961 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7eef9-b767-46fc-9363-1ddca168caec admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:54.890129 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7eef9-b767-46fc-9363-1ddca168caec admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:54.890317 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7eef9-b767-46fc-9363-1ddca168caec admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:54.891116 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7eef9-b767-46fc-9363-1ddca168caec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:54.892734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7eef9-b767-46fc-9363-1ddca168caec admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:54.905511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1004/2008] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:18:54.917889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e116e-b8df-48e8-9fbf-73e6692564ef admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:54.930030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e116e-b8df-48e8-9fbf-73e6692564ef admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:54.936616 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8a8df-843d-4591-920b-a4caf255a809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:54.938703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8a8df-843d-4591-920b-a4caf255a809 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:54.939215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8a8df-843d-4591-920b-a4caf255a809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:54.939750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8a8df-843d-4591-920b-a4caf255a809 admin admin] PATH_INFO: `/v3/projects/4527ff8365ac407085e35fe07e530bba/users/fbce3eaa17c54629bee157bb7620c74c/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:54.943874 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8a8df-843d-4591-920b-a4caf255a809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4527ff8365ac407085e35fe07e530bba, user_id=fbce3eaa17c54629bee157bb7620c74c, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:54.954760 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8a8df-843d-4591-920b-a4caf255a809 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:54.973508 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1005/2009] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:18:54 2022] PUT /identity/v3/projects/4527ff8365ac407085e35fe07e530bba/users/fbce3eaa17c54629bee157bb7620c74c/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:18:54.985561 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60001985-13d9-4421-a0f7-890d3dd895ec None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:54.985951 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60001985-13d9-4421-a0f7-890d3dd895ec None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:54.986312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60001985-13d9-4421-a0f7-890d3dd895ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:55.015949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-60001985-13d9-4421-a0f7-890d3dd895ec None None] MFA Rules not processed for user `fbce3eaa17c54629bee157bb7620c74c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:55.017161 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-60001985-13d9-4421-a0f7-890d3dd895ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:55.108519 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1005/2010] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:54 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:55.121889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8a8df-843d-4591-920b-a4caf255a809 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:55.124420 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-abc8a8df-843d-4591-920b-a4caf255a809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:55.200451 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8a8df-843d-4591-920b-a4caf255a809 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:55.206076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea62c27-0089-4c8f-9da2-607c6eb10180 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:55.208311 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea62c27-0089-4c8f-9da2-607c6eb10180 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:55.209171 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea62c27-0089-4c8f-9da2-607c6eb10180 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:55.209646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea62c27-0089-4c8f-9da2-607c6eb10180 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:55.229624 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea62c27-0089-4c8f-9da2-607c6eb10180 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:55.231583 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea62c27-0089-4c8f-9da2-607c6eb10180 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:55.243427 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1006/2011] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:55 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:55.353902 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60001985-13d9-4421-a0f7-890d3dd895ec None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:55.355609 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-60001985-13d9-4421-a0f7-890d3dd895ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:55.438313 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60001985-13d9-4421-a0f7-890d3dd895ec None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:55.443694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9a03e-4257-41f5-b6cc-ad56e068835a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:55.444791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9a03e-4257-41f5-b6cc-ad56e068835a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:55.445149 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9a03e-4257-41f5-b6cc-ad56e068835a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:55.450857 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9a03e-4257-41f5-b6cc-ad56e068835a admin admin] PATH_INFO: `/v3/users/1ff41b6fdf254d1599564870b2073940` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:55.450857 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9a03e-4257-41f5-b6cc-ad56e068835a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ff41b6fdf254d1599564870b2073940)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:55.450857 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9a03e-4257-41f5-b6cc-ad56e068835a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:55.662868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:55.675781 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-49b9a03e-4257-41f5-b6cc-ad56e068835a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ff41b6fdf254d1599564870b2073940'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:55.694570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-49b9a03e-4257-41f5-b6cc-ad56e068835a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ff41b6fdf254d1599564870b2073940'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:56.477798 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-49b9a03e-4257-41f5-b6cc-ad56e068835a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ff41b6fdf254d1599564870b2073940'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:56.478498 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-49b9a03e-4257-41f5-b6cc-ad56e068835a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ff41b6fdf254d1599564870b2073940'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:56.479558 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-49b9a03e-4257-41f5-b6cc-ad56e068835a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ff41b6fdf254d1599564870b2073940'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:56.502529 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1006/2012] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:55 2022] DELETE /identity/v3/users/1ff41b6fdf254d1599564870b2073940 => generated 0 bytes in 1146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:56.504743 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dab55-3c30-4695-8911-dd5e39f9fb36 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:56.505534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dab55-3c30-4695-8911-dd5e39f9fb36 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:56.506002 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217dab55-3c30-4695-8911-dd5e39f9fb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:56.531693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-217dab55-3c30-4695-8911-dd5e39f9fb36 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:56.532911 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-217dab55-3c30-4695-8911-dd5e39f9fb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:56.603638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1007/2013] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:56.614775 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9a03e-4257-41f5-b6cc-ad56e068835a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:56.616305 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-49b9a03e-4257-41f5-b6cc-ad56e068835a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:56.699551 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9a03e-4257-41f5-b6cc-ad56e068835a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:56.704639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b9ee4-7760-4169-b3f1-d9614b87ec7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:56.705612 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b9ee4-7760-4169-b3f1-d9614b87ec7e service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:56.706013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b9ee4-7760-4169-b3f1-d9614b87ec7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:56.706510 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b9ee4-7760-4169-b3f1-d9614b87ec7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:56.719227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b9ee4-7760-4169-b3f1-d9614b87ec7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:56.721389 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b9ee4-7760-4169-b3f1-d9614b87ec7e service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:56.734228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1007/2014] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:18:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:18:57.446102 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dab55-3c30-4695-8911-dd5e39f9fb36 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:57.447826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-217dab55-3c30-4695-8911-dd5e39f9fb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:57.471832 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217dab55-3c30-4695-8911-dd5e39f9fb36 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:57.488452 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88a450-5969-4982-8b62-c09289ee9b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:57.490027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88a450-5969-4982-8b62-c09289ee9b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:57.491343 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88a450-5969-4982-8b62-c09289ee9b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:57.491684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88a450-5969-4982-8b62-c09289ee9b7d admin admin] PATH_INFO: `/v3/projects/8724cc790d8d4c01b2ab642574728dcd` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:57.493982 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88a450-5969-4982-8b62-c09289ee9b7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8724cc790d8d4c01b2ab642574728dcd)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:57.497026 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88a450-5969-4982-8b62-c09289ee9b7d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:57.660124 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:57.662632 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ec88a450-5969-4982-8b62-c09289ee9b7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8724cc790d8d4c01b2ab642574728dcd', 'request_id': 'req-ec88a450-5969-4982-8b62-c09289ee9b7d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:57.674091 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ec88a450-5969-4982-8b62-c09289ee9b7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8724cc790d8d4c01b2ab642574728dcd', 'request_id': 'req-ec88a450-5969-4982-8b62-c09289ee9b7d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:57.688648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ec88a450-5969-4982-8b62-c09289ee9b7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8724cc790d8d4c01b2ab642574728dcd', 'request_id': 'req-ec88a450-5969-4982-8b62-c09289ee9b7d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:57.804273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ec88a450-5969-4982-8b62-c09289ee9b7d admin admin] The token cache is being invalidate because project 8724cc790d8d4c01b2ab642574728dcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:57.804847 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ec88a450-5969-4982-8b62-c09289ee9b7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:57.805727 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ec88a450-5969-4982-8b62-c09289ee9b7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:57.808217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1008/2015] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:57 2022] DELETE /identity/v3/projects/8724cc790d8d4c01b2ab642574728dcd => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:57.812535 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b9ee4-7760-4169-b3f1-d9614b87ec7e service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:57.825088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b9ee4-7760-4169-b3f1-d9614b87ec7e service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:57.912203 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4007009-4621-4667-9f9d-73cc61dae992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:57.913130 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4007009-4621-4667-9f9d-73cc61dae992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:57.913421 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4007009-4621-4667-9f9d-73cc61dae992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:57.913703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4007009-4621-4667-9f9d-73cc61dae992 admin admin] PATH_INFO: `/v3/users/df856445e86a4edbb25689a2b5cea3b9` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:57.915094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4007009-4621-4667-9f9d-73cc61dae992 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df856445e86a4edbb25689a2b5cea3b9)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:57.920358 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4007009-4621-4667-9f9d-73cc61dae992 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:58.163378 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c4007009-4621-4667-9f9d-73cc61dae992 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df856445e86a4edbb25689a2b5cea3b9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:58.177727 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:58.179206 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c4007009-4621-4667-9f9d-73cc61dae992 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df856445e86a4edbb25689a2b5cea3b9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:58.244689 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c4007009-4621-4667-9f9d-73cc61dae992 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df856445e86a4edbb25689a2b5cea3b9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:58.249870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c4007009-4621-4667-9f9d-73cc61dae992 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df856445e86a4edbb25689a2b5cea3b9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:58.250080 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c4007009-4621-4667-9f9d-73cc61dae992 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df856445e86a4edbb25689a2b5cea3b9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:58.305524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1008/2016] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:57 2022] DELETE /identity/v3/users/df856445e86a4edbb25689a2b5cea3b9 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:58.420619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675a14b-f0fe-46bc-93a6-df79a9554531 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:58.422000 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675a14b-f0fe-46bc-93a6-df79a9554531 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:58.422512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675a14b-f0fe-46bc-93a6-df79a9554531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:58.449221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9675a14b-f0fe-46bc-93a6-df79a9554531 None None] MFA Rules not processed for user `98f58f91d17949198e66883d234a9e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:58.451269 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9675a14b-f0fe-46bc-93a6-df79a9554531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:58.546331 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1009/2017] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jun 24 04:18:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:58.611047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4007009-4621-4667-9f9d-73cc61dae992 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:58.612490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c4007009-4621-4667-9f9d-73cc61dae992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:58.726509 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4007009-4621-4667-9f9d-73cc61dae992 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:58.734322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2ddb6-2c9a-4897-bbf1-3b2a9c86483e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8e1f1e3c7e42cc9fa6f82aaac55129', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:58.735758 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2ddb6-2c9a-4897-bbf1-3b2a9c86483e service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:58.736049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2ddb6-2c9a-4897-bbf1-3b2a9c86483e service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:58.736870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2ddb6-2c9a-4897-bbf1-3b2a9c86483e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:58.755704 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2ddb6-2c9a-4897-bbf1-3b2a9c86483e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:58.757156 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2ddb6-2c9a-4897-bbf1-3b2a9c86483e service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:58.782101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1009/2018] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jun 24 04:18:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 04:18:58.954828 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675a14b-f0fe-46bc-93a6-df79a9554531 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:58.956488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9675a14b-f0fe-46bc-93a6-df79a9554531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:59.067669 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675a14b-f0fe-46bc-93a6-df79a9554531 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:59.072384 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b785fc-e8ce-4715-bf43-5d9df76ea045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:59.073257 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b785fc-e8ce-4715-bf43-5d9df76ea045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:59.073535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b785fc-e8ce-4715-bf43-5d9df76ea045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:59.073999 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b785fc-e8ce-4715-bf43-5d9df76ea045 admin admin] PATH_INFO: `/v3/projects/2bd67c36893f40bfbd60177ddf2a2483` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:59.075319 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b785fc-e8ce-4715-bf43-5d9df76ea045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bd67c36893f40bfbd60177ddf2a2483)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:59.078550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b785fc-e8ce-4715-bf43-5d9df76ea045 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:59.241034 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-80b785fc-e8ce-4715-bf43-5d9df76ea045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bd67c36893f40bfbd60177ddf2a2483', 'request_id': 'req-80b785fc-e8ce-4715-bf43-5d9df76ea045'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:59.250304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-80b785fc-e8ce-4715-bf43-5d9df76ea045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bd67c36893f40bfbd60177ddf2a2483', 'request_id': 'req-80b785fc-e8ce-4715-bf43-5d9df76ea045'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:59.263390 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-80b785fc-e8ce-4715-bf43-5d9df76ea045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bd67c36893f40bfbd60177ddf2a2483', 'request_id': 'req-80b785fc-e8ce-4715-bf43-5d9df76ea045'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:59.350234 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-80b785fc-e8ce-4715-bf43-5d9df76ea045 admin admin] The token cache is being invalidate because project 2bd67c36893f40bfbd60177ddf2a2483 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:18:59.350595 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-80b785fc-e8ce-4715-bf43-5d9df76ea045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:59.351232 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-80b785fc-e8ce-4715-bf43-5d9df76ea045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:18:59.352793 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1010/2019] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:18:58 2022] DELETE /identity/v3/projects/2bd67c36893f40bfbd60177ddf2a2483 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:18:59.358413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af658c58-79ec-425a-8321-7bb2fe41f262 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:59.358904 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af658c58-79ec-425a-8321-7bb2fe41f262 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:59.358993 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af658c58-79ec-425a-8321-7bb2fe41f262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:59.378982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-af658c58-79ec-425a-8321-7bb2fe41f262 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:59.380481 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-af658c58-79ec-425a-8321-7bb2fe41f262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:59.461020 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1010/2020] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:59.472672 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576604d3-127d-4465-80bf-ffaf2b466ca7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:59.472672 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576604d3-127d-4465-80bf-ffaf2b466ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:59.472672 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576604d3-127d-4465-80bf-ffaf2b466ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:59.515545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-576604d3-127d-4465-80bf-ffaf2b466ca7 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:18:59.517188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-576604d3-127d-4465-80bf-ffaf2b466ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:59.548050 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1011/2021] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:18:59.552581 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af658c58-79ec-425a-8321-7bb2fe41f262 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:59.571973 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af658c58-79ec-425a-8321-7bb2fe41f262 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:59.579932 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fdf17e-a655-4746-8299-1b7bf657c456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:59.580956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fdf17e-a655-4746-8299-1b7bf657c456 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:59.581592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fdf17e-a655-4746-8299-1b7bf657c456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:59.582060 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fdf17e-a655-4746-8299-1b7bf657c456 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:59.583391 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fdf17e-a655-4746-8299-1b7bf657c456 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:18:59.583973 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fdf17e-a655-4746-8299-1b7bf657c456 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:59.586597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fdf17e-a655-4746-8299-1b7bf657c456 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:59.608146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1011/2022] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:18:59.612960 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576604d3-127d-4465-80bf-ffaf2b466ca7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:59.627825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576604d3-127d-4465-80bf-ffaf2b466ca7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:59.658713 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6212d-0980-4452-8f4d-6f8c88932131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:59.659613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6212d-0980-4452-8f4d-6f8c88932131 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:59.660074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6212d-0980-4452-8f4d-6f8c88932131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:59.660556 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6212d-0980-4452-8f4d-6f8c88932131 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:59.661274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fdf17e-a655-4746-8299-1b7bf657c456 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:59.661732 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6212d-0980-4452-8f4d-6f8c88932131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:59.663203 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-34fdf17e-a655-4746-8299-1b7bf657c456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:18:59.664294 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6212d-0980-4452-8f4d-6f8c88932131 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:59.692429 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fdf17e-a655-4746-8299-1b7bf657c456 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:59.698259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9c86c-8123-4e87-92b1-0a48af1d193a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:59.699806 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9c86c-8123-4e87-92b1-0a48af1d193a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:59.699955 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9c86c-8123-4e87-92b1-0a48af1d193a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:59.700282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9c86c-8123-4e87-92b1-0a48af1d193a admin admin] PATH_INFO: `/v3/users/ef679b01efe4428aa4dad0b3af8af5ac` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:59.703065 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9c86c-8123-4e87-92b1-0a48af1d193a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef679b01efe4428aa4dad0b3af8af5ac)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:59.707227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9c86c-8123-4e87-92b1-0a48af1d193a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:18:59.779537 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1012/2023] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:18:59 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:18:59.784927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6212d-0980-4452-8f4d-6f8c88932131 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:18:59.852673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:18:59.894713 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6212d-0980-4452-8f4d-6f8c88932131 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:18:59.915920 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b1494-c6e0-4ac8-a010-5832037eb1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:18:59.916720 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b1494-c6e0-4ac8-a010-5832037eb1bf admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:18:59.917097 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b1494-c6e0-4ac8-a010-5832037eb1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:18:59.917484 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b1494-c6e0-4ac8-a010-5832037eb1bf admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:18:59.920965 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b1494-c6e0-4ac8-a010-5832037eb1bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:18:59.923958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b1494-c6e0-4ac8-a010-5832037eb1bf admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:00.026778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-59f9c86c-8123-4e87-92b1-0a48af1d193a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef679b01efe4428aa4dad0b3af8af5ac'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:00.043117 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1013/2024] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:18:59 2022] POST /identity/v3/users => generated 289 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:00.047421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b1494-c6e0-4ac8-a010-5832037eb1bf admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:00.063066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-59f9c86c-8123-4e87-92b1-0a48af1d193a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef679b01efe4428aa4dad0b3af8af5ac'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:00.068638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b1494-c6e0-4ac8-a010-5832037eb1bf admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:00.076986 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c49ee4-e700-42da-9823-a18f4861f7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:00.077682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c49ee4-e700-42da-9823-a18f4861f7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:00.077965 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c49ee4-e700-42da-9823-a18f4861f7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:00.078238 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c49ee4-e700-42da-9823-a18f4861f7b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:00.079027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c49ee4-e700-42da-9823-a18f4861f7b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:00.080668 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c49ee4-e700-42da-9823-a18f4861f7b9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:00.099528 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1014/2025] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:00.100492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c49ee4-e700-42da-9823-a18f4861f7b9 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:00.119477 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c49ee4-e700-42da-9823-a18f4861f7b9 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:00.128576 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc537641-6668-4dda-9f46-fba29d3b3be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:00.129699 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc537641-6668-4dda-9f46-fba29d3b3be6 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:00.130219 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc537641-6668-4dda-9f46-fba29d3b3be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:00.130708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc537641-6668-4dda-9f46-fba29d3b3be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:00.132061 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc537641-6668-4dda-9f46-fba29d3b3be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:00.134146 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc537641-6668-4dda-9f46-fba29d3b3be6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:00.152321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1015/2026] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:00.152321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def371e3-a6e3-4797-b269-7436ae5f10a6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:00.152321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def371e3-a6e3-4797-b269-7436ae5f10a6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:00.152647 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def371e3-a6e3-4797-b269-7436ae5f10a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:00.158751 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-59f9c86c-8123-4e87-92b1-0a48af1d193a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef679b01efe4428aa4dad0b3af8af5ac'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:00.159500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-59f9c86c-8123-4e87-92b1-0a48af1d193a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef679b01efe4428aa4dad0b3af8af5ac'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:00.160539 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-59f9c86c-8123-4e87-92b1-0a48af1d193a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef679b01efe4428aa4dad0b3af8af5ac'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:00.176857 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-def371e3-a6e3-4797-b269-7436ae5f10a6 None None] MFA Rules not processed for user `98f58f91d17949198e66883d234a9e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:00.178161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-def371e3-a6e3-4797-b269-7436ae5f10a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:00.183836 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1012/2027] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:18:59 2022] DELETE /identity/v3/users/ef679b01efe4428aa4dad0b3af8af5ac => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:19:00.183836 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9c86c-8123-4e87-92b1-0a48af1d193a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:00.184706 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-59f9c86c-8123-4e87-92b1-0a48af1d193a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:00.251536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:00.255855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1016/2028] 10.222.0.16 () {58 vars in 967 bytes} [Fri Jun 24 04:19:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:00.267827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a23d55-28f0-4be0-b740-457325ed5e5f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:00.268397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a23d55-28f0-4be0-b740-457325ed5e5f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:00.268758 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a23d55-28f0-4be0-b740-457325ed5e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:00.285802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9c86c-8123-4e87-92b1-0a48af1d193a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:00.292423 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76e403-3a8d-4ef5-abef-1c1f51a96c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:00.294122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76e403-3a8d-4ef5-abef-1c1f51a96c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:00.294909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76e403-3a8d-4ef5-abef-1c1f51a96c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:00.295567 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76e403-3a8d-4ef5-abef-1c1f51a96c0a admin admin] PATH_INFO: `/v3/projects/b33acd01b27b41208c338588719e0acf/users/e1563be888b74ee789955eb285cd7376/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:00.302845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-69a23d55-28f0-4be0-b740-457325ed5e5f None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:00.305295 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-69a23d55-28f0-4be0-b740-457325ed5e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:00.326190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76e403-3a8d-4ef5-abef-1c1f51a96c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b33acd01b27b41208c338588719e0acf, user_id=e1563be888b74ee789955eb285cd7376, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:00.326190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76e403-3a8d-4ef5-abef-1c1f51a96c0a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:00.326482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1017/2029] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:00.332580 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a23d55-28f0-4be0-b740-457325ed5e5f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:00.334591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-69a23d55-28f0-4be0-b740-457325ed5e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:00.418481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a23d55-28f0-4be0-b740-457325ed5e5f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:00.424918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55749cba-7f8d-4803-9843-6056e241ad19 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8e1f1e3c7e42cc9fa6f82aaac55129', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:00.426100 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55749cba-7f8d-4803-9843-6056e241ad19 service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:00.434314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1013/2030] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:00 2022] PUT /identity/v3/projects/b33acd01b27b41208c338588719e0acf/users/e1563be888b74ee789955eb285cd7376/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:00.436466 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76e403-3a8d-4ef5-abef-1c1f51a96c0a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:00.437610 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55749cba-7f8d-4803-9843-6056e241ad19 service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:00.437925 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55749cba-7f8d-4803-9843-6056e241ad19 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:00.439981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bb76e403-3a8d-4ef5-abef-1c1f51a96c0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:00.452993 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55749cba-7f8d-4803-9843-6056e241ad19 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:00.455046 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55749cba-7f8d-4803-9843-6056e241ad19 service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:00.485795 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1018/2031] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jun 24 04:19:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 04:19:00.487373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f4772-f22a-4b88-acf4-25950619c050 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:00.488865 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f4772-f22a-4b88-acf4-25950619c050 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:00.488865 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f4772-f22a-4b88-acf4-25950619c050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:00.530508 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-369f4772-f22a-4b88-acf4-25950619c050 None None] MFA Rules not processed for user `e1563be888b74ee789955eb285cd7376`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:00.531799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-369f4772-f22a-4b88-acf4-25950619c050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:00.549317 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76e403-3a8d-4ef5-abef-1c1f51a96c0a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:00.554411 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9956d62-a348-4762-b166-4b20b4fd7283 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:00.555078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9956d62-a348-4762-b166-4b20b4fd7283 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:00.555487 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9956d62-a348-4762-b166-4b20b4fd7283 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:00.555644 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9956d62-a348-4762-b166-4b20b4fd7283 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:00.624341 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1019/2032] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:00.628689 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f4772-f22a-4b88-acf4-25950619c050 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:00.651746 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9956d62-a348-4762-b166-4b20b4fd7283 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:00.652279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9956d62-a348-4762-b166-4b20b4fd7283 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:00.665090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1014/2033] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:19:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:19:00.728461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f4772-f22a-4b88-acf4-25950619c050 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:00.738739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26752f6-c906-49ea-8712-c0e8a6e9ad7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:00.739802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26752f6-c906-49ea-8712-c0e8a6e9ad7b admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:00.740425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26752f6-c906-49ea-8712-c0e8a6e9ad7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:00.740939 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26752f6-c906-49ea-8712-c0e8a6e9ad7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:00.741969 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26752f6-c906-49ea-8712-c0e8a6e9ad7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:00.744375 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26752f6-c906-49ea-8712-c0e8a6e9ad7b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:00.858293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1020/2034] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:19:00 2022] POST /identity/v3/projects => generated 377 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:00.861803 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9956d62-a348-4762-b166-4b20b4fd7283 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:00.873710 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9956d62-a348-4762-b166-4b20b4fd7283 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:00.948699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ca7c6-b72b-4738-9578-dc4593dd9f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:00.949284 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ca7c6-b72b-4738-9578-dc4593dd9f9a admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:00.949876 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ca7c6-b72b-4738-9578-dc4593dd9f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:00.949876 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ca7c6-b72b-4738-9578-dc4593dd9f9a admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:00.951112 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ca7c6-b72b-4738-9578-dc4593dd9f9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:00.953229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ca7c6-b72b-4738-9578-dc4593dd9f9a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:01.074137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1015/2035] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:19:00 2022] POST /identity/v3/users => generated 289 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:01.081566 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26752f6-c906-49ea-8712-c0e8a6e9ad7b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:01.100981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26752f6-c906-49ea-8712-c0e8a6e9ad7b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:01.105820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0daa6-e78a-4149-82dd-7fac43faf6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:01.106539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0daa6-e78a-4149-82dd-7fac43faf6ad admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:01.107509 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0daa6-e78a-4149-82dd-7fac43faf6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:01.107816 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0daa6-e78a-4149-82dd-7fac43faf6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:01.108570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0daa6-e78a-4149-82dd-7fac43faf6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:01.110208 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0daa6-e78a-4149-82dd-7fac43faf6ad admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:01.137670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1021/2036] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:01.139765 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ca7c6-b72b-4738-9578-dc4593dd9f9a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:01.170445 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ca7c6-b72b-4738-9578-dc4593dd9f9a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:01.170445 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679bb69-663c-4c38-b23d-8a84ce32953a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:01.170445 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679bb69-663c-4c38-b23d-8a84ce32953a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:01.170445 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679bb69-663c-4c38-b23d-8a84ce32953a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:01.170827 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679bb69-663c-4c38-b23d-8a84ce32953a admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:01.170827 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679bb69-663c-4c38-b23d-8a84ce32953a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:01.170827 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679bb69-663c-4c38-b23d-8a84ce32953a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:01.188244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0daa6-e78a-4149-82dd-7fac43faf6ad admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:01.195290 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1016/2037] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:01.201774 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0daa6-e78a-4149-82dd-7fac43faf6ad admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:01.207688 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90acf7e1-0c67-40ad-91d1-b9de82ef923a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:01.208733 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90acf7e1-0c67-40ad-91d1-b9de82ef923a admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:01.209841 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90acf7e1-0c67-40ad-91d1-b9de82ef923a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:01.210134 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90acf7e1-0c67-40ad-91d1-b9de82ef923a admin admin] PATH_INFO: `/v3/projects/ca84616386aa40a78c20e0613e31997c/users/c2ddbf73041546fa92a55b5889473812/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:01.227638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90acf7e1-0c67-40ad-91d1-b9de82ef923a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca84616386aa40a78c20e0613e31997c, user_id=c2ddbf73041546fa92a55b5889473812, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:01.230599 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90acf7e1-0c67-40ad-91d1-b9de82ef923a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:01.530174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d59c2-8287-49ee-a9a3-71355c891e53 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:01.530174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d59c2-8287-49ee-a9a3-71355c891e53 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:01.530174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d59c2-8287-49ee-a9a3-71355c891e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:01.530174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b49d59c2-8287-49ee-a9a3-71355c891e53 None None] MFA Rules not processed for user `98f58f91d17949198e66883d234a9e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:01.530174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b49d59c2-8287-49ee-a9a3-71355c891e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:01.533072 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1017/2038] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jun 24 04:19:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:19:01.546857 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d59c2-8287-49ee-a9a3-71355c891e53 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:01.609307 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1022/2039] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:01 2022] PUT /identity/v3/projects/ca84616386aa40a78c20e0613e31997c/users/c2ddbf73041546fa92a55b5889473812/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:01.618180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90acf7e1-0c67-40ad-91d1-b9de82ef923a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:01.624794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-90acf7e1-0c67-40ad-91d1-b9de82ef923a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:01.639922 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d59c2-8287-49ee-a9a3-71355c891e53 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:01.731479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90acf7e1-0c67-40ad-91d1-b9de82ef923a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:01.732625 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559e4c2-83c7-4e4f-9281-5d6b2f9f7b56 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8e1f1e3c7e42cc9fa6f82aaac55129', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:01.733702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559e4c2-83c7-4e4f-9281-5d6b2f9f7b56 service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:01.734154 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559e4c2-83c7-4e4f-9281-5d6b2f9f7b56 service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:01.734677 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559e4c2-83c7-4e4f-9281-5d6b2f9f7b56 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:01.743102 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71576cda-9074-47a0-86b7-3085062a71a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:01.744554 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71576cda-9074-47a0-86b7-3085062a71a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:01.744852 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71576cda-9074-47a0-86b7-3085062a71a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:01.745139 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71576cda-9074-47a0-86b7-3085062a71a1 admin admin] PATH_INFO: `/v3/projects/0c2337d47cc14154bb1ec9d768f8aad9` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:01.746962 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71576cda-9074-47a0-86b7-3085062a71a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c2337d47cc14154bb1ec9d768f8aad9)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:01.751527 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71576cda-9074-47a0-86b7-3085062a71a1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:01.823750 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559e4c2-83c7-4e4f-9281-5d6b2f9f7b56 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:01.827418 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559e4c2-83c7-4e4f-9281-5d6b2f9f7b56 service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:01.839784 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1018/2040] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jun 24 04:19:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 24 04:19:01.841958 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e37122-b717-4f6e-84a5-fe3b26544dba None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:01.842324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e37122-b717-4f6e-84a5-fe3b26544dba None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:01.842598 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e37122-b717-4f6e-84a5-fe3b26544dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:01.880704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-71576cda-9074-47a0-86b7-3085062a71a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c2337d47cc14154bb1ec9d768f8aad9', 'request_id': 'req-71576cda-9074-47a0-86b7-3085062a71a1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:01.889882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-23e37122-b717-4f6e-84a5-fe3b26544dba None None] MFA Rules not processed for user `c2ddbf73041546fa92a55b5889473812`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:01.892316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-23e37122-b717-4f6e-84a5-fe3b26544dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:01.894784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-71576cda-9074-47a0-86b7-3085062a71a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c2337d47cc14154bb1ec9d768f8aad9', 'request_id': 'req-71576cda-9074-47a0-86b7-3085062a71a1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:01.911067 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-71576cda-9074-47a0-86b7-3085062a71a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c2337d47cc14154bb1ec9d768f8aad9', 'request_id': 'req-71576cda-9074-47a0-86b7-3085062a71a1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:02.040381 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-71576cda-9074-47a0-86b7-3085062a71a1 admin admin] The token cache is being invalidate because project 0c2337d47cc14154bb1ec9d768f8aad9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:19:02.040748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-71576cda-9074-47a0-86b7-3085062a71a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:02.042346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-71576cda-9074-47a0-86b7-3085062a71a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:02.048807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1023/2041] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:19:01 2022] DELETE /identity/v3/projects/0c2337d47cc14154bb1ec9d768f8aad9 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:02.049815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71576cda-9074-47a0-86b7-3085062a71a1 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:02.054075 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1019/2042] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:02.054250 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-71576cda-9074-47a0-86b7-3085062a71a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:02.055450 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e37122-b717-4f6e-84a5-fe3b26544dba None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:02.068514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:02.189498 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e37122-b717-4f6e-84a5-fe3b26544dba None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:02.196276 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71576cda-9074-47a0-86b7-3085062a71a1 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:02.202528 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419e027-a8af-4bd6-8afb-cdf21dc506c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:02.203308 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419e027-a8af-4bd6-8afb-cdf21dc506c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:02.203708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419e027-a8af-4bd6-8afb-cdf21dc506c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:02.204082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419e027-a8af-4bd6-8afb-cdf21dc506c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:02.204933 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419e027-a8af-4bd6-8afb-cdf21dc506c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:02.205618 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2425cad-5350-4181-abcc-3195c3695033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:02.207469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2425cad-5350-4181-abcc-3195c3695033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:02.207929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2425cad-5350-4181-abcc-3195c3695033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:02.208318 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419e027-a8af-4bd6-8afb-cdf21dc506c8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:02.208713 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2425cad-5350-4181-abcc-3195c3695033 admin admin] PATH_INFO: `/v3/users/fbce3eaa17c54629bee157bb7620c74c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:02.210811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2425cad-5350-4181-abcc-3195c3695033 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbce3eaa17c54629bee157bb7620c74c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:02.214587 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2425cad-5350-4181-abcc-3195c3695033 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:02.344022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1020/2043] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:19:02 2022] POST /identity/v3/projects => generated 377 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:02.345860 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419e027-a8af-4bd6-8afb-cdf21dc506c8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:02.357333 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3419e027-a8af-4bd6-8afb-cdf21dc506c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:02.453889 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419e027-a8af-4bd6-8afb-cdf21dc506c8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:02.463316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b41344-fd44-41d0-8aab-50dd8be0f07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:02.464020 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b41344-fd44-41d0-8aab-50dd8be0f07d admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:02.464305 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b41344-fd44-41d0-8aab-50dd8be0f07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:02.464584 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b41344-fd44-41d0-8aab-50dd8be0f07d admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:02.465394 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b41344-fd44-41d0-8aab-50dd8be0f07d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:02.468816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b41344-fd44-41d0-8aab-50dd8be0f07d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:02.475658 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d2425cad-5350-4181-abcc-3195c3695033 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbce3eaa17c54629bee157bb7620c74c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:02.510157 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d2425cad-5350-4181-abcc-3195c3695033 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbce3eaa17c54629bee157bb7620c74c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:02.510970 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d2425cad-5350-4181-abcc-3195c3695033 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbce3eaa17c54629bee157bb7620c74c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:02.511511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:02.511812 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d2425cad-5350-4181-abcc-3195c3695033 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbce3eaa17c54629bee157bb7620c74c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:02.520879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d2425cad-5350-4181-abcc-3195c3695033 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbce3eaa17c54629bee157bb7620c74c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:02.560469 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1021/2044] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:19:02 2022] POST /identity/v3/users => generated 295 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:02.561218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a115182-1508-402b-86eb-1588aef3c060 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:02.561947 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a115182-1508-402b-86eb-1588aef3c060 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:02.562154 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a115182-1508-402b-86eb-1588aef3c060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:02.592447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7a115182-1508-402b-86eb-1588aef3c060 None None] MFA Rules not processed for user `f3821e46c09b449d85c46c58d99bf607`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:02.593936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7a115182-1508-402b-86eb-1588aef3c060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:02.614688 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1024/2045] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:19:02 2022] DELETE /identity/v3/users/fbce3eaa17c54629bee157bb7620c74c => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:02.615300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2425cad-5350-4181-abcc-3195c3695033 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:02.616693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d2425cad-5350-4181-abcc-3195c3695033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:02.723949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2425cad-5350-4181-abcc-3195c3695033 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:02.728143 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1022/2046] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:02.728143 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd89d0c-889b-4240-b0ed-3be7be5ab2db None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:02.728143 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd89d0c-889b-4240-b0ed-3be7be5ab2db None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:02.728143 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd89d0c-889b-4240-b0ed-3be7be5ab2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:02.730004 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e59d0-0aa8-4714-8852-6c836a31a93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:02.730777 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e59d0-0aa8-4714-8852-6c836a31a93b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:02.731066 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e59d0-0aa8-4714-8852-6c836a31a93b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:02.731482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e59d0-0aa8-4714-8852-6c836a31a93b admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:02.733075 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e59d0-0aa8-4714-8852-6c836a31a93b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:02.735395 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e59d0-0aa8-4714-8852-6c836a31a93b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:02.746439 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1025/2047] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:02.747156 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e59d0-0aa8-4714-8852-6c836a31a93b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:02.750943 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c17e59d0-0aa8-4714-8852-6c836a31a93b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:02.751470 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ebd89d0c-889b-4240-b0ed-3be7be5ab2db None None] MFA Rules not processed for user `98f58f91d17949198e66883d234a9e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:02.752753 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ebd89d0c-889b-4240-b0ed-3be7be5ab2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:02.776477 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e59d0-0aa8-4714-8852-6c836a31a93b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:02.783331 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc1918-5937-470f-87bf-93c197f32080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:02.784092 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc1918-5937-470f-87bf-93c197f32080 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:02.784377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc1918-5937-470f-87bf-93c197f32080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:02.784653 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc1918-5937-470f-87bf-93c197f32080 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:02.785367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc1918-5937-470f-87bf-93c197f32080 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:02.787411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc1918-5937-470f-87bf-93c197f32080 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:02.832607 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1023/2048] 10.222.0.16 () {58 vars in 967 bytes} [Fri Jun 24 04:19:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:02.833858 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd89d0c-889b-4240-b0ed-3be7be5ab2db None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:02.846237 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd89d0c-889b-4240-b0ed-3be7be5ab2db None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:02.850992 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e49536-36af-4c03-aa9a-86ab01860424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:02.851922 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e49536-36af-4c03-aa9a-86ab01860424 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:02.852210 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e49536-36af-4c03-aa9a-86ab01860424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:02.852486 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e49536-36af-4c03-aa9a-86ab01860424 admin admin] PATH_INFO: `/v3/projects/daf93111b68d48c490ed1c1d733fc3a1/users/b26cede92240458aa227a9b30b26f538/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:02.867822 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e49536-36af-4c03-aa9a-86ab01860424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daf93111b68d48c490ed1c1d733fc3a1, user_id=b26cede92240458aa227a9b30b26f538, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:02.871097 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e49536-36af-4c03-aa9a-86ab01860424 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:02.915309 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1026/2049] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:19:02 2022] POST /identity/v3/projects => generated 371 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:02.915309 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc1918-5937-470f-87bf-93c197f32080 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:02.915309 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-39fc1918-5937-470f-87bf-93c197f32080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:02.971571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1024/2050] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:02 2022] PUT /identity/v3/projects/daf93111b68d48c490ed1c1d733fc3a1/users/b26cede92240458aa227a9b30b26f538/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:02.971571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e49536-36af-4c03-aa9a-86ab01860424 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:03.013612 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc1918-5937-470f-87bf-93c197f32080 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:03.053350 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e49536-36af-4c03-aa9a-86ab01860424 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:03.058930 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918b62d-3a45-48bd-9c1a-a6f3f2983f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:03.059771 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918b62d-3a45-48bd-9c1a-a6f3f2983f13 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:03.060854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918b62d-3a45-48bd-9c1a-a6f3f2983f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:03.061178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918b62d-3a45-48bd-9c1a-a6f3f2983f13 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:03.061955 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918b62d-3a45-48bd-9c1a-a6f3f2983f13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:03.063384 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:03.064056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918b62d-3a45-48bd-9c1a-a6f3f2983f13 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:03.093391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c292f-6676-4669-a8c3-edda0e043404 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8e1f1e3c7e42cc9fa6f82aaac55129', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:03.094606 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c292f-6676-4669-a8c3-edda0e043404 service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:03.101762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c292f-6676-4669-a8c3-edda0e043404 service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:03.101762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c292f-6676-4669-a8c3-edda0e043404 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:03.182286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c292f-6676-4669-a8c3-edda0e043404 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:03.183948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c292f-6676-4669-a8c3-edda0e043404 service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:03.194054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1027/2051] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jun 24 04:19:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 04:19:03.195872 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c292f-6676-4669-a8c3-edda0e043404 service swift] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:03.209736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1025/2052] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:19:02 2022] POST /identity/v3/users => generated 292 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:03.209836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c292f-6676-4669-a8c3-edda0e043404 service swift] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:03.219753 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918b62d-3a45-48bd-9c1a-a6f3f2983f13 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:03.223078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d918b62d-3a45-48bd-9c1a-a6f3f2983f13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:03.223819 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3c02a-4466-43a9-8d84-109c7c2d615b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:03.225437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3c02a-4466-43a9-8d84-109c7c2d615b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:03.226237 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3c02a-4466-43a9-8d84-109c7c2d615b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:03.228761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3c02a-4466-43a9-8d84-109c7c2d615b admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:03.229517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3c02a-4466-43a9-8d84-109c7c2d615b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:03.231418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3c02a-4466-43a9-8d84-109c7c2d615b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:03.241246 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918b62d-3a45-48bd-9c1a-a6f3f2983f13 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:03.247367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1028/2053] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:03.247367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3c02a-4466-43a9-8d84-109c7c2d615b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:03.247674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb6a50-abf6-4773-be65-36ffe5ab2e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:03.247674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb6a50-abf6-4773-be65-36ffe5ab2e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:03.247674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb6a50-abf6-4773-be65-36ffe5ab2e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:03.248019 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb6a50-abf6-4773-be65-36ffe5ab2e6e admin admin] PATH_INFO: `/v3/projects/4527ff8365ac407085e35fe07e530bba` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:03.248975 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb6a50-abf6-4773-be65-36ffe5ab2e6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4527ff8365ac407085e35fe07e530bba)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:03.251064 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb6a50-abf6-4773-be65-36ffe5ab2e6e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:03.260216 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3c02a-4466-43a9-8d84-109c7c2d615b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:03.268574 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7276f4-ff74-4128-b188-4622adb98218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:03.269254 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7276f4-ff74-4128-b188-4622adb98218 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:03.269542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7276f4-ff74-4128-b188-4622adb98218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:03.269824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7276f4-ff74-4128-b188-4622adb98218 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:03.270608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7276f4-ff74-4128-b188-4622adb98218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:03.272509 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7276f4-ff74-4128-b188-4622adb98218 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:03.286723 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1029/2054] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:03.288030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7276f4-ff74-4128-b188-4622adb98218 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:03.303502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7276f4-ff74-4128-b188-4622adb98218 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:03.310989 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aaa358-5ffb-4fa3-98f1-095a2573118d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:03.312398 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aaa358-5ffb-4fa3-98f1-095a2573118d admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:03.312883 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aaa358-5ffb-4fa3-98f1-095a2573118d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:03.313620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aaa358-5ffb-4fa3-98f1-095a2573118d admin admin] PATH_INFO: `/v3/projects/daf93111b68d48c490ed1c1d733fc3a1/users/b26cede92240458aa227a9b30b26f538/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:03.315256 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aaa358-5ffb-4fa3-98f1-095a2573118d admin admin] RBAC: Authorizing `identity:create_grant(project_id=daf93111b68d48c490ed1c1d733fc3a1, user_id=b26cede92240458aa227a9b30b26f538, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:03.318190 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aaa358-5ffb-4fa3-98f1-095a2573118d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:03.330697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1030/2055] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:03 2022] PUT /identity/v3/projects/daf93111b68d48c490ed1c1d733fc3a1/users/b26cede92240458aa227a9b30b26f538/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:03.335299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aaa358-5ffb-4fa3-98f1-095a2573118d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:03.367144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-35bb6a50-abf6-4773-be65-36ffe5ab2e6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4527ff8365ac407085e35fe07e530bba', 'request_id': 'req-35bb6a50-abf6-4773-be65-36ffe5ab2e6e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:03.414154 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aaa358-5ffb-4fa3-98f1-095a2573118d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:03.462595 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-35bb6a50-abf6-4773-be65-36ffe5ab2e6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4527ff8365ac407085e35fe07e530bba', 'request_id': 'req-35bb6a50-abf6-4773-be65-36ffe5ab2e6e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:03.469949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-35bb6a50-abf6-4773-be65-36ffe5ab2e6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4527ff8365ac407085e35fe07e530bba', 'request_id': 'req-35bb6a50-abf6-4773-be65-36ffe5ab2e6e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:03.485158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-35bb6a50-abf6-4773-be65-36ffe5ab2e6e admin admin] The token cache is being invalidate because project 4527ff8365ac407085e35fe07e530bba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:19:03.485446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-35bb6a50-abf6-4773-be65-36ffe5ab2e6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:03.486332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-35bb6a50-abf6-4773-be65-36ffe5ab2e6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:03.488343 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1026/2056] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:19:03 2022] DELETE /identity/v3/projects/4527ff8365ac407085e35fe07e530bba => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:03.499874 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff4d06-69b5-4a56-aeda-689cd2aa3dee None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:03.501420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff4d06-69b5-4a56-aeda-689cd2aa3dee None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:03.501702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff4d06-69b5-4a56-aeda-689cd2aa3dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:03.510775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9cfcd-8be4-4034-b417-614fbc3d0005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:03.512110 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9cfcd-8be4-4034-b417-614fbc3d0005 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:03.512484 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9cfcd-8be4-4034-b417-614fbc3d0005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:03.513477 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9cfcd-8be4-4034-b417-614fbc3d0005 admin admin] PATH_INFO: `/v3/projects/34ebafeecaa04e3196f815aff6a6f00d/users/ba662104accd44a1aa5e7a473bb51557/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:03.536137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9cfcd-8be4-4034-b417-614fbc3d0005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34ebafeecaa04e3196f815aff6a6f00d, user_id=ba662104accd44a1aa5e7a473bb51557, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:03.538972 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9cfcd-8be4-4034-b417-614fbc3d0005 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:03.547689 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-5dff4d06-69b5-4a56-aeda-689cd2aa3dee None None] MFA Rules not processed for user `b26cede92240458aa227a9b30b26f538`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:03.548976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5dff4d06-69b5-4a56-aeda-689cd2aa3dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:03.629264 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:03.636467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1031/2057] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:03 2022] PUT /identity/v3/projects/34ebafeecaa04e3196f815aff6a6f00d/users/ba662104accd44a1aa5e7a473bb51557/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:03.637527 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95730bf-c86e-4f3c-8569-1fc65cdf8692 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:03.645684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95730bf-c86e-4f3c-8569-1fc65cdf8692 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:03.645684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95730bf-c86e-4f3c-8569-1fc65cdf8692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:03.646672 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1027/2058] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:03 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:03.646672 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff4d06-69b5-4a56-aeda-689cd2aa3dee None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:03.648057 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5dff4d06-69b5-4a56-aeda-689cd2aa3dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:03.670631 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff4d06-69b5-4a56-aeda-689cd2aa3dee None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:03.675282 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d95730bf-c86e-4f3c-8569-1fc65cdf8692 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:03.677786 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d95730bf-c86e-4f3c-8569-1fc65cdf8692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:03.720486 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:03.757678 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4b511-609b-4e8f-b1a1-7b23e9b361f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:03.758571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4b511-609b-4e8f-b1a1-7b23e9b361f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:03.759088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4b511-609b-4e8f-b1a1-7b23e9b361f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:03.759637 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4b511-609b-4e8f-b1a1-7b23e9b361f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:03.760398 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef4b511-609b-4e8f-b1a1-7b23e9b361f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:03.762021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef4b511-609b-4e8f-b1a1-7b23e9b361f5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:03.783516 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1032/2059] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:03.790500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1028/2060] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:19:03.790646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95730bf-c86e-4f3c-8569-1fc65cdf8692 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:03.790646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d95730bf-c86e-4f3c-8569-1fc65cdf8692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:03.790814 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4b511-609b-4e8f-b1a1-7b23e9b361f5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:03.800529 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4b511-609b-4e8f-b1a1-7b23e9b361f5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:03.811919 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38de6f96-c79a-4e43-883b-d087837ee3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:03.811919 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de6f96-c79a-4e43-883b-d087837ee3dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:03.811919 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de6f96-c79a-4e43-883b-d087837ee3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:03.811919 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38de6f96-c79a-4e43-883b-d087837ee3dc admin admin] PATH_INFO: `/v3/projects/34ebafeecaa04e3196f815aff6a6f00d/users/ba662104accd44a1aa5e7a473bb51557/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:03.816384 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de6f96-c79a-4e43-883b-d087837ee3dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=34ebafeecaa04e3196f815aff6a6f00d, user_id=ba662104accd44a1aa5e7a473bb51557, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:03.819727 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38de6f96-c79a-4e43-883b-d087837ee3dc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:03.842173 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1029/2061] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:03 2022] PUT /identity/v3/projects/34ebafeecaa04e3196f815aff6a6f00d/users/ba662104accd44a1aa5e7a473bb51557/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:19:03.844056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a8d6b-bdbe-45d3-954a-f1243e6a1128 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:03.844302 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a8d6b-bdbe-45d3-954a-f1243e6a1128 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:03.844973 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a8d6b-bdbe-45d3-954a-f1243e6a1128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:03.875959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a43a8d6b-bdbe-45d3-954a-f1243e6a1128 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:03.875959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a43a8d6b-bdbe-45d3-954a-f1243e6a1128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:03.904738 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95730bf-c86e-4f3c-8569-1fc65cdf8692 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:04.026660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1030/2062] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:04.031878 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a8d6b-bdbe-45d3-954a-f1243e6a1128 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:04.058947 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a8d6b-bdbe-45d3-954a-f1243e6a1128 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:04.062033 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:04.077088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e40866-8f81-4b6c-8736-ed1f486d79f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:04.077895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e40866-8f81-4b6c-8736-ed1f486d79f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:04.078241 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e40866-8f81-4b6c-8736-ed1f486d79f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:04.078547 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e40866-8f81-4b6c-8736-ed1f486d79f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:04.079294 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e40866-8f81-4b6c-8736-ed1f486d79f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:19:04.083088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e40866-8f81-4b6c-8736-ed1f486d79f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:04.083088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e40866-8f81-4b6c-8736-ed1f486d79f3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:04.105010 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1031/2063] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:19:04.105146 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069c847-9532-4849-88b7-e75c1afb5012 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:04.106151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069c847-9532-4849-88b7-e75c1afb5012 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:04.107340 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069c847-9532-4849-88b7-e75c1afb5012 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:04.107825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069c847-9532-4849-88b7-e75c1afb5012 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:04.109880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e40866-8f81-4b6c-8736-ed1f486d79f3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:04.123242 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5069c847-9532-4849-88b7-e75c1afb5012 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:04.123427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e40866-8f81-4b6c-8736-ed1f486d79f3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:04.125736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00cba9-7d75-4c29-b0eb-34410e24ba6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:04.126510 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00cba9-7d75-4c29-b0eb-34410e24ba6b admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:04.126802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00cba9-7d75-4c29-b0eb-34410e24ba6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:04.127082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00cba9-7d75-4c29-b0eb-34410e24ba6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:04.128227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00cba9-7d75-4c29-b0eb-34410e24ba6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:04.129860 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00cba9-7d75-4c29-b0eb-34410e24ba6b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:04.210304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069c847-9532-4849-88b7-e75c1afb5012 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:04.212224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069c847-9532-4849-88b7-e75c1afb5012 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:04.224028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1033/2064] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:19:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:19:04.249622 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1032/2065] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:19:04 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:04.250644 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069c847-9532-4849-88b7-e75c1afb5012 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:04.264091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:04.267099 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069c847-9532-4849-88b7-e75c1afb5012 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:04.355138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd2e00-b32e-4c28-a5b0-330e861c1470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:04.355138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd2e00-b32e-4c28-a5b0-330e861c1470 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:04.355138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd2e00-b32e-4c28-a5b0-330e861c1470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:04.355138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd2e00-b32e-4c28-a5b0-330e861c1470 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:04.355498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd2e00-b32e-4c28-a5b0-330e861c1470 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:04.358428 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd2e00-b32e-4c28-a5b0-330e861c1470 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:04.365795 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:04.508482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1034/2066] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:19:04 2022] POST /identity/v3/users => generated 294 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:19:04.511556 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00cba9-7d75-4c29-b0eb-34410e24ba6b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:04.526527 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00cba9-7d75-4c29-b0eb-34410e24ba6b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:04.539135 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f668b34-4193-4deb-84a1-60ff7c700292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:04.542777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f668b34-4193-4deb-84a1-60ff7c700292 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:04.542777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f668b34-4193-4deb-84a1-60ff7c700292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:04.542777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f668b34-4193-4deb-84a1-60ff7c700292 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:04.542777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f668b34-4193-4deb-84a1-60ff7c700292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:04.545070 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f668b34-4193-4deb-84a1-60ff7c700292 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:04.557912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1033/2067] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:04.563162 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd2e00-b32e-4c28-a5b0-330e861c1470 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:04.573751 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd2e00-b32e-4c28-a5b0-330e861c1470 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:04.578929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb4be7-9786-4007-ba26-cb8da65ef1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:04.579721 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb4be7-9786-4007-ba26-cb8da65ef1b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:04.580073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb4be7-9786-4007-ba26-cb8da65ef1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:04.580428 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb4be7-9786-4007-ba26-cb8da65ef1b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:04.581177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb4be7-9786-4007-ba26-cb8da65ef1b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:04.583073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb4be7-9786-4007-ba26-cb8da65ef1b1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:04.596550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1035/2068] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:04.600645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f668b34-4193-4deb-84a1-60ff7c700292 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:04.614461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f668b34-4193-4deb-84a1-60ff7c700292 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:04.624167 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb3f48-f5b9-4829-86b7-e1de7f23e5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:04.625669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb3f48-f5b9-4829-86b7-e1de7f23e5ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:04.625669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb3f48-f5b9-4829-86b7-e1de7f23e5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:04.626127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb3f48-f5b9-4829-86b7-e1de7f23e5ce admin admin] PATH_INFO: `/v3/projects/23d24f03380a40259f20a7f57e18dbb3/users/0f1fcf4dcb194248a32d7edd48f773b1/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:04.656630 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb3f48-f5b9-4829-86b7-e1de7f23e5ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=23d24f03380a40259f20a7f57e18dbb3, user_id=0f1fcf4dcb194248a32d7edd48f773b1, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:04.660622 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb3f48-f5b9-4829-86b7-e1de7f23e5ce admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:04.776231 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1034/2069] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:04 2022] PUT /identity/v3/projects/23d24f03380a40259f20a7f57e18dbb3/users/0f1fcf4dcb194248a32d7edd48f773b1/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:04.776425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf36db5-3179-4cbd-b008-13b0727d8a6a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:04.776674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf36db5-3179-4cbd-b008-13b0727d8a6a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:04.776900 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf36db5-3179-4cbd-b008-13b0727d8a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:04.816214 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4cf36db5-3179-4cbd-b008-13b0727d8a6a None None] MFA Rules not processed for user `0f1fcf4dcb194248a32d7edd48f773b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:04.818025 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4cf36db5-3179-4cbd-b008-13b0727d8a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:04.918449 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1036/2070] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:04.920397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb3f48-f5b9-4829-86b7-e1de7f23e5ce admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:04.944744 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:05.126929 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb3f48-f5b9-4829-86b7-e1de7f23e5ce admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:05.151140 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c24c0-e045-4190-9deb-26fce4456627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:05.152656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c24c0-e045-4190-9deb-26fce4456627 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:05.158938 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c24c0-e045-4190-9deb-26fce4456627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:05.159359 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c24c0-e045-4190-9deb-26fce4456627 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:05.160209 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c24c0-e045-4190-9deb-26fce4456627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:05.163406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c24c0-e045-4190-9deb-26fce4456627 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:05.307719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1035/2071] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:19:04 2022] POST /identity/v3/projects => generated 385 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:05.313896 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf36db5-3179-4cbd-b008-13b0727d8a6a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:05.334999 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf36db5-3179-4cbd-b008-13b0727d8a6a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:05.443943 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d740a2-a3f9-447a-aaaf-8e893f628783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:05.444523 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d740a2-a3f9-447a-aaaf-8e893f628783 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:05.444884 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d740a2-a3f9-447a-aaaf-8e893f628783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:05.445355 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d740a2-a3f9-447a-aaaf-8e893f628783 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:05.446390 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d740a2-a3f9-447a-aaaf-8e893f628783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:05.451101 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d740a2-a3f9-447a-aaaf-8e893f628783 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:05.612933 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1037/2072] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:19:05 2022] POST /identity/v3/users => generated 299 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:05.616004 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c24c0-e045-4190-9deb-26fce4456627 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:05.626528 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c24c0-e045-4190-9deb-26fce4456627 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:05.632185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d793d-00cc-487f-9c59-40873fdd077f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:05.633011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d793d-00cc-487f-9c59-40873fdd077f admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:05.633365 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d793d-00cc-487f-9c59-40873fdd077f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:05.643192 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d793d-00cc-487f-9c59-40873fdd077f admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:05.643192 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d793d-00cc-487f-9c59-40873fdd077f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:05.643192 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d793d-00cc-487f-9c59-40873fdd077f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:05.650694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1036/2073] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:05.661841 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d740a2-a3f9-447a-aaaf-8e893f628783 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:05.663535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:05.675989 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d740a2-a3f9-447a-aaaf-8e893f628783 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:05.682136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b27b88-be44-4836-99a0-1fcdb84c8f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:05.683402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b27b88-be44-4836-99a0-1fcdb84c8f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:05.683691 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b27b88-be44-4836-99a0-1fcdb84c8f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:05.683974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b27b88-be44-4836-99a0-1fcdb84c8f34 admin admin] PATH_INFO: `/v3/projects/42dbb802602d4199a90fd6b7c945ea05/users/123363693d594df08aa372e8edab7478/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:05.701518 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b27b88-be44-4836-99a0-1fcdb84c8f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42dbb802602d4199a90fd6b7c945ea05, user_id=123363693d594df08aa372e8edab7478, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:05.704373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b27b88-be44-4836-99a0-1fcdb84c8f34 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:05.840257 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1038/2074] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:05 2022] PUT /identity/v3/projects/42dbb802602d4199a90fd6b7c945ea05/users/123363693d594df08aa372e8edab7478/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:05.844508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d793d-00cc-487f-9c59-40873fdd077f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:05.859297 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d793d-00cc-487f-9c59-40873fdd077f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:05.992272 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a3f2a-fbad-4bc5-9dff-ea86f74cc996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:05.993198 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a3f2a-fbad-4bc5-9dff-ea86f74cc996 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:05.993654 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a3f2a-fbad-4bc5-9dff-ea86f74cc996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:05.994136 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a3f2a-fbad-4bc5-9dff-ea86f74cc996 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:05.995184 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a3f2a-fbad-4bc5-9dff-ea86f74cc996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:05.996965 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a3f2a-fbad-4bc5-9dff-ea86f74cc996 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:06.018439 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1037/2075] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:06.020887 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b27b88-be44-4836-99a0-1fcdb84c8f34 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:06.039294 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b27b88-be44-4836-99a0-1fcdb84c8f34 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:06.048123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3223bd-f452-4cb3-b35e-9243a211e934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:06.053120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3223bd-f452-4cb3-b35e-9243a211e934 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:06.053585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3223bd-f452-4cb3-b35e-9243a211e934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:06.054033 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3223bd-f452-4cb3-b35e-9243a211e934 admin admin] PATH_INFO: `/v3/projects/42dbb802602d4199a90fd6b7c945ea05/users/123363693d594df08aa372e8edab7478/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:06.061869 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3223bd-f452-4cb3-b35e-9243a211e934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42dbb802602d4199a90fd6b7c945ea05, user_id=123363693d594df08aa372e8edab7478, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:06.065731 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3223bd-f452-4cb3-b35e-9243a211e934 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:06.081464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1039/2076] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:06 2022] PUT /identity/v3/projects/42dbb802602d4199a90fd6b7c945ea05/users/123363693d594df08aa372e8edab7478/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:06.092270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee512c4-4e38-4287-b64e-e33b44392188 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:06.094867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee512c4-4e38-4287-b64e-e33b44392188 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:06.094867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee512c4-4e38-4287-b64e-e33b44392188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:06.121996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-cee512c4-4e38-4287-b64e-e33b44392188 None None] MFA Rules not processed for user `123363693d594df08aa372e8edab7478`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:06.124465 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-cee512c4-4e38-4287-b64e-e33b44392188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:06.220744 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1038/2077] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:06.231148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3223bd-f452-4cb3-b35e-9243a211e934 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:06.371859 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3223bd-f452-4cb3-b35e-9243a211e934 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:06.376984 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b2ed4-6755-4de7-ba20-56085f259fa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:06.377700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b2ed4-6755-4de7-ba20-56085f259fa3 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:06.377987 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b2ed4-6755-4de7-ba20-56085f259fa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:06.378270 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b2ed4-6755-4de7-ba20-56085f259fa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:06.388774 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b2ed4-6755-4de7-ba20-56085f259fa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:06.390270 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b2ed4-6755-4de7-ba20-56085f259fa3 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:06.400634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1040/2078] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:19:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:19:06.499624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee512c4-4e38-4287-b64e-e33b44392188 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:06.639708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee512c4-4e38-4287-b64e-e33b44392188 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:06.648676 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd5fffc-970b-4798-b0f8-4eecfe0648b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:06.651023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5fffc-970b-4798-b0f8-4eecfe0648b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:06.654063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5fffc-970b-4798-b0f8-4eecfe0648b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:06.654063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5fffc-970b-4798-b0f8-4eecfe0648b6 admin admin] PATH_INFO: `/v3/users/0f1fcf4dcb194248a32d7edd48f773b1` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:06.654490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5fffc-970b-4798-b0f8-4eecfe0648b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f1fcf4dcb194248a32d7edd48f773b1)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:06.657314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd5fffc-970b-4798-b0f8-4eecfe0648b6 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:06.704921 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b2ed4-6755-4de7-ba20-56085f259fa3 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:06.706449 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2e8b2ed4-6755-4de7-ba20-56085f259fa3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:06.708623 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:06.738031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b2ed4-6755-4de7-ba20-56085f259fa3 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:06.748768 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b7ce2-a7a1-4f54-8af6-94fec0cb382f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:06.750453 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b7ce2-a7a1-4f54-8af6-94fec0cb382f service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:06.751037 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b7ce2-a7a1-4f54-8af6-94fec0cb382f service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:06.751373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99b7ce2-a7a1-4f54-8af6-94fec0cb382f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:06.753953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a99b7ce2-a7a1-4f54-8af6-94fec0cb382f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:06.836299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b7ce2-a7a1-4f54-8af6-94fec0cb382f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:06.838046 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99b7ce2-a7a1-4f54-8af6-94fec0cb382f service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:06.864152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1041/2079] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:19:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:19:06.865248 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b7ce2-a7a1-4f54-8af6-94fec0cb382f service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:06.867946 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a99b7ce2-a7a1-4f54-8af6-94fec0cb382f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:06.896520 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99b7ce2-a7a1-4f54-8af6-94fec0cb382f service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:06.900157 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:06.909855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e48a7-b1fb-4ef8-94f8-0c8be0d9e2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:06.911012 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e48a7-b1fb-4ef8-94f8-0c8be0d9e2c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:06.912849 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e48a7-b1fb-4ef8-94f8-0c8be0d9e2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:06.913705 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e48a7-b1fb-4ef8-94f8-0c8be0d9e2c2 admin admin] PATH_INFO: `/v3/users/9dbe4952ee034749b771ff380da6e06c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:06.917912 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e48a7-b1fb-4ef8-94f8-0c8be0d9e2c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dbe4952ee034749b771ff380da6e06c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:06.917912 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e48a7-b1fb-4ef8-94f8-0c8be0d9e2c2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:06.963320 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ddd5fffc-970b-4798-b0f8-4eecfe0648b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f1fcf4dcb194248a32d7edd48f773b1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:06.977425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ddd5fffc-970b-4798-b0f8-4eecfe0648b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f1fcf4dcb194248a32d7edd48f773b1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:07.070922 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ddd5fffc-970b-4798-b0f8-4eecfe0648b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f1fcf4dcb194248a32d7edd48f773b1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:07.071868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ddd5fffc-970b-4798-b0f8-4eecfe0648b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f1fcf4dcb194248a32d7edd48f773b1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:07.072713 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ddd5fffc-970b-4798-b0f8-4eecfe0648b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f1fcf4dcb194248a32d7edd48f773b1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:07.090192 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1039/2080] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:19:06 2022] DELETE /identity/v3/users/0f1fcf4dcb194248a32d7edd48f773b1 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:07.096138 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c2cd4-9585-4313-94e7-f1beac376c05 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:07.098179 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c2cd4-9585-4313-94e7-f1beac376c05 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:07.098687 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c2cd4-9585-4313-94e7-f1beac376c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:07.133223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-350c2cd4-9585-4313-94e7-f1beac376c05 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:07.134681 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-350c2cd4-9585-4313-94e7-f1beac376c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:07.166232 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-017e48a7-b1fb-4ef8-94f8-0c8be0d9e2c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dbe4952ee034749b771ff380da6e06c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:07.181454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-017e48a7-b1fb-4ef8-94f8-0c8be0d9e2c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dbe4952ee034749b771ff380da6e06c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:07.182220 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-017e48a7-b1fb-4ef8-94f8-0c8be0d9e2c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dbe4952ee034749b771ff380da6e06c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:07.182914 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-017e48a7-b1fb-4ef8-94f8-0c8be0d9e2c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dbe4952ee034749b771ff380da6e06c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:07.192873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-017e48a7-b1fb-4ef8-94f8-0c8be0d9e2c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dbe4952ee034749b771ff380da6e06c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:07.203161 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1040/2081] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:07.214789 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c2cd4-9585-4313-94e7-f1beac376c05 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:07.217600 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-350c2cd4-9585-4313-94e7-f1beac376c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:07.302662 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1042/2082] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:19:06 2022] DELETE /identity/v3/users/9dbe4952ee034749b771ff380da6e06c => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:07.307130 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c9b73-ab6b-494c-bd9e-f4b5126c0631 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:07.307900 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c9b73-ab6b-494c-bd9e-f4b5126c0631 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:07.308287 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c9b73-ab6b-494c-bd9e-f4b5126c0631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:07.316696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c2cd4-9585-4313-94e7-f1beac376c05 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:07.329268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-083c9b73-ab6b-494c-bd9e-f4b5126c0631 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:07.330708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-083c9b73-ab6b-494c-bd9e-f4b5126c0631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:07.392630 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19934d-d10d-49ef-a47a-0bade7935b61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:07.394899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19934d-d10d-49ef-a47a-0bade7935b61 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:07.396696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19934d-d10d-49ef-a47a-0bade7935b61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:07.396696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19934d-d10d-49ef-a47a-0bade7935b61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:07.420499 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1043/2083] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:07.435148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c9b73-ab6b-494c-bd9e-f4b5126c0631 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:07.437526 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-083c9b73-ab6b-494c-bd9e-f4b5126c0631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:07.452928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c9b73-ab6b-494c-bd9e-f4b5126c0631 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:07.459315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a37ab6-f886-41b3-8de0-4dd2aacb752d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:07.461046 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a37ab6-f886-41b3-8de0-4dd2aacb752d service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:07.461523 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a37ab6-f886-41b3-8de0-4dd2aacb752d service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:07.462152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a37ab6-f886-41b3-8de0-4dd2aacb752d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:07.475353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a37ab6-f886-41b3-8de0-4dd2aacb752d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:07.476824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a37ab6-f886-41b3-8de0-4dd2aacb752d service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:07.490350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1044/2084] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:19:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:19:07.516142 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19934d-d10d-49ef-a47a-0bade7935b61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:07.519657 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19934d-d10d-49ef-a47a-0bade7935b61 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:07.538169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1041/2085] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:19:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:19:07.821226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:07.888612 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a37ab6-f886-41b3-8de0-4dd2aacb752d service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:07.889599 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-59a37ab6-f886-41b3-8de0-4dd2aacb752d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:07.914180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a37ab6-f886-41b3-8de0-4dd2aacb752d service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:07.923105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70a2d6-a0dd-45d6-bd0d-9e7140ff4eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:07.924329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70a2d6-a0dd-45d6-bd0d-9e7140ff4eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:07.924617 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70a2d6-a0dd-45d6-bd0d-9e7140ff4eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:07.924901 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70a2d6-a0dd-45d6-bd0d-9e7140ff4eb5 admin admin] PATH_INFO: `/v3/projects/09b6462b30a34629bda3cfb3b7270c6d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:07.927755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70a2d6-a0dd-45d6-bd0d-9e7140ff4eb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09b6462b30a34629bda3cfb3b7270c6d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:07.931150 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70a2d6-a0dd-45d6-bd0d-9e7140ff4eb5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:08.125450 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ff70a2d6-a0dd-45d6-bd0d-9e7140ff4eb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09b6462b30a34629bda3cfb3b7270c6d', 'request_id': 'req-ff70a2d6-a0dd-45d6-bd0d-9e7140ff4eb5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:08.133176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ff70a2d6-a0dd-45d6-bd0d-9e7140ff4eb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09b6462b30a34629bda3cfb3b7270c6d', 'request_id': 'req-ff70a2d6-a0dd-45d6-bd0d-9e7140ff4eb5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:08.141831 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ff70a2d6-a0dd-45d6-bd0d-9e7140ff4eb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09b6462b30a34629bda3cfb3b7270c6d', 'request_id': 'req-ff70a2d6-a0dd-45d6-bd0d-9e7140ff4eb5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:08.213125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19934d-d10d-49ef-a47a-0bade7935b61 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:08.219082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1c19934d-d10d-49ef-a47a-0bade7935b61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:08.236246 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:08.242670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ff70a2d6-a0dd-45d6-bd0d-9e7140ff4eb5 admin admin] The token cache is being invalidate because project 09b6462b30a34629bda3cfb3b7270c6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:19:08.243225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ff70a2d6-a0dd-45d6-bd0d-9e7140ff4eb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:08.244266 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ff70a2d6-a0dd-45d6-bd0d-9e7140ff4eb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:08.246834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1045/2086] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:19:07 2022] DELETE /identity/v3/projects/09b6462b30a34629bda3cfb3b7270c6d => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:08.251822 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a14b7-3f75-4abe-9871-81ae33376626 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:08.252401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a14b7-3f75-4abe-9871-81ae33376626 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:08.252910 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a14b7-3f75-4abe-9871-81ae33376626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:08.253073 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19934d-d10d-49ef-a47a-0bade7935b61 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:08.306689 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-033a14b7-3f75-4abe-9871-81ae33376626 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:08.308002 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-033a14b7-3f75-4abe-9871-81ae33376626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:08.345689 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff8a62-8e6a-4855-8153-a154eee6721f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:08.346989 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff8a62-8e6a-4855-8153-a154eee6721f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:08.347500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff8a62-8e6a-4855-8153-a154eee6721f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:08.347998 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff8a62-8e6a-4855-8153-a154eee6721f admin admin] PATH_INFO: `/v3/projects/23d24f03380a40259f20a7f57e18dbb3` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:08.350192 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff8a62-8e6a-4855-8153-a154eee6721f admin admin] RBAC: Authorizing `identity:delete_project(project_id=23d24f03380a40259f20a7f57e18dbb3)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:08.353016 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff8a62-8e6a-4855-8153-a154eee6721f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:08.438630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1046/2087] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:08.445663 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62f905-c57f-4919-beae-2e9442d819be None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:08.445663 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62f905-c57f-4919-beae-2e9442d819be None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:08.445663 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62f905-c57f-4919-beae-2e9442d819be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:08.468279 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4b62f905-c57f-4919-beae-2e9442d819be None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:08.469742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4b62f905-c57f-4919-beae-2e9442d819be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:08.478608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1047/2088] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:08.486395 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62f905-c57f-4919-beae-2e9442d819be None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:08.496170 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62f905-c57f-4919-beae-2e9442d819be None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:08.501270 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d49f8-cc60-441f-be3e-1dfecaaa3ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:08.508644 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d49f8-cc60-441f-be3e-1dfecaaa3ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:08.508644 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d49f8-cc60-441f-be3e-1dfecaaa3ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:08.508644 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d49f8-cc60-441f-be3e-1dfecaaa3ab2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:08.508644 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d49f8-cc60-441f-be3e-1dfecaaa3ab2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:19:08.508644 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d49f8-cc60-441f-be3e-1dfecaaa3ab2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:08.508644 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d49f8-cc60-441f-be3e-1dfecaaa3ab2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:08.519288 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1048/2089] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:19:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:19:08.523979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d49f8-cc60-441f-be3e-1dfecaaa3ab2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:08.535454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d49f8-cc60-441f-be3e-1dfecaaa3ab2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:08.541088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2bcd5-1f82-4ccb-ac91-73c6b749a910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:08.541770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2bcd5-1f82-4ccb-ac91-73c6b749a910 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:08.542062 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2bcd5-1f82-4ccb-ac91-73c6b749a910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:08.542344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2bcd5-1f82-4ccb-ac91-73c6b749a910 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:08.543100 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2bcd5-1f82-4ccb-ac91-73c6b749a910 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:08.545203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2bcd5-1f82-4ccb-ac91-73c6b749a910 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:08.580242 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-44ff8a62-8e6a-4855-8153-a154eee6721f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23d24f03380a40259f20a7f57e18dbb3', 'request_id': 'req-44ff8a62-8e6a-4855-8153-a154eee6721f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:08.685300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1049/2090] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:19:08 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:08.686181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2bcd5-1f82-4ccb-ac91-73c6b749a910 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:08.725362 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-44ff8a62-8e6a-4855-8153-a154eee6721f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23d24f03380a40259f20a7f57e18dbb3', 'request_id': 'req-44ff8a62-8e6a-4855-8153-a154eee6721f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:08.734196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-44ff8a62-8e6a-4855-8153-a154eee6721f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23d24f03380a40259f20a7f57e18dbb3', 'request_id': 'req-44ff8a62-8e6a-4855-8153-a154eee6721f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:08.752446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-44ff8a62-8e6a-4855-8153-a154eee6721f admin admin] The token cache is being invalidate because project 23d24f03380a40259f20a7f57e18dbb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:19:08.752824 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-44ff8a62-8e6a-4855-8153-a154eee6721f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:08.754503 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-44ff8a62-8e6a-4855-8153-a154eee6721f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:08.769277 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1042/2091] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:19:08 2022] DELETE /identity/v3/projects/23d24f03380a40259f20a7f57e18dbb3 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:19:08.769277 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff8a62-8e6a-4855-8153-a154eee6721f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:08.792497 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-44ff8a62-8e6a-4855-8153-a154eee6721f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:08.811111 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2bcd5-1f82-4ccb-ac91-73c6b749a910 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:08.817414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a388fd3-aeb3-45df-ab9d-5f58f9a2caca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:08.818268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a388fd3-aeb3-45df-ab9d-5f58f9a2caca admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:08.819200 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a388fd3-aeb3-45df-ab9d-5f58f9a2caca admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:08.819512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a388fd3-aeb3-45df-ab9d-5f58f9a2caca admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:08.820197 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a388fd3-aeb3-45df-ab9d-5f58f9a2caca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:08.822854 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a388fd3-aeb3-45df-ab9d-5f58f9a2caca admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:08.923855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1050/2092] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:19:08 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:19:08.929013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a388fd3-aeb3-45df-ab9d-5f58f9a2caca admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:08.939274 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3a388fd3-aeb3-45df-ab9d-5f58f9a2caca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:08.957191 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff8a62-8e6a-4855-8153-a154eee6721f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:08.961988 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a388fd3-aeb3-45df-ab9d-5f58f9a2caca admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:08.963804 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37982bc-dfa9-4ddd-a3f0-d5fda22ca78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:08.964915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37982bc-dfa9-4ddd-a3f0-d5fda22ca78d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:08.965375 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37982bc-dfa9-4ddd-a3f0-d5fda22ca78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:08.965901 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37982bc-dfa9-4ddd-a3f0-d5fda22ca78d admin admin] PATH_INFO: `/v3/users/123363693d594df08aa372e8edab7478` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:08.968457 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37982bc-dfa9-4ddd-a3f0-d5fda22ca78d admin admin] RBAC: Authorizing `identity:delete_user(user_id=123363693d594df08aa372e8edab7478)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:08.971539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ffb61-eafa-4fac-a5cd-194ab69678e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:08.972233 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ffb61-eafa-4fac-a5cd-194ab69678e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:08.972355 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37982bc-dfa9-4ddd-a3f0-d5fda22ca78d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:08.972522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ffb61-eafa-4fac-a5cd-194ab69678e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:08.972803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ffb61-eafa-4fac-a5cd-194ab69678e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:08.973547 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ffb61-eafa-4fac-a5cd-194ab69678e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:08.975268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ffb61-eafa-4fac-a5cd-194ab69678e4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:09.008333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1051/2093] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:09.009595 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ffb61-eafa-4fac-a5cd-194ab69678e4 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:09.013011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-565ffb61-eafa-4fac-a5cd-194ab69678e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:09.033558 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:09.090560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ffb61-eafa-4fac-a5cd-194ab69678e4 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:09.095544 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1423d3-bc27-479e-965a-61c24586f7b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:09.096484 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1423d3-bc27-479e-965a-61c24586f7b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:09.096916 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1423d3-bc27-479e-965a-61c24586f7b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:09.097369 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1423d3-bc27-479e-965a-61c24586f7b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:09.099833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-cd1423d3-bc27-479e-965a-61c24586f7b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:09.184228 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1423d3-bc27-479e-965a-61c24586f7b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:09.185988 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1423d3-bc27-479e-965a-61c24586f7b9 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:09.197143 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1052/2094] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:19:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:19:09.198038 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1423d3-bc27-479e-965a-61c24586f7b9 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:09.209397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1423d3-bc27-479e-965a-61c24586f7b9 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:09.217773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304eaf98-05fa-41c6-af49-d2b2849b956b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:09.217773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304eaf98-05fa-41c6-af49-d2b2849b956b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:09.218046 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304eaf98-05fa-41c6-af49-d2b2849b956b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:09.218221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304eaf98-05fa-41c6-af49-d2b2849b956b admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:09.218943 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304eaf98-05fa-41c6-af49-d2b2849b956b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:09.220609 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304eaf98-05fa-41c6-af49-d2b2849b956b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:09.230745 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b37982bc-dfa9-4ddd-a3f0-d5fda22ca78d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '123363693d594df08aa372e8edab7478'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:09.233195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1053/2095] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:09.237792 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304eaf98-05fa-41c6-af49-d2b2849b956b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:09.251292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304eaf98-05fa-41c6-af49-d2b2849b956b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:09.255233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b37982bc-dfa9-4ddd-a3f0-d5fda22ca78d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '123363693d594df08aa372e8edab7478'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:09.258695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdff0f-5a1e-43c8-bf39-1b4318caca07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:09.260306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdff0f-5a1e-43c8-bf39-1b4318caca07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:09.260702 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdff0f-5a1e-43c8-bf39-1b4318caca07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:09.261120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cdff0f-5a1e-43c8-bf39-1b4318caca07 admin admin] PATH_INFO: `/v3/projects/d51a251ca7084b2bb07d808026cccd67/users/4fb36d1e057543a3abcf6d6c2564d9ae/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:09.282478 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cdff0f-5a1e-43c8-bf39-1b4318caca07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d51a251ca7084b2bb07d808026cccd67, user_id=4fb36d1e057543a3abcf6d6c2564d9ae, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:09.285313 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cdff0f-5a1e-43c8-bf39-1b4318caca07 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:09.347655 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b37982bc-dfa9-4ddd-a3f0-d5fda22ca78d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '123363693d594df08aa372e8edab7478'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:09.348602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b37982bc-dfa9-4ddd-a3f0-d5fda22ca78d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '123363693d594df08aa372e8edab7478'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:09.349269 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b37982bc-dfa9-4ddd-a3f0-d5fda22ca78d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '123363693d594df08aa372e8edab7478'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:09.363027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1043/2096] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:19:08 2022] DELETE /identity/v3/users/123363693d594df08aa372e8edab7478 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:19:09.411610 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1054/2097] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:09 2022] PUT /identity/v3/projects/d51a251ca7084b2bb07d808026cccd67/users/4fb36d1e057543a3abcf6d6c2564d9ae/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:19:09.416930 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8aff8-f31b-4486-9a72-93f1eff1cfaf None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:09.417431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8aff8-f31b-4486-9a72-93f1eff1cfaf None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:09.418141 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8aff8-f31b-4486-9a72-93f1eff1cfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:09.450760 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1ee8aff8-f31b-4486-9a72-93f1eff1cfaf None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:09.453732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1ee8aff8-f31b-4486-9a72-93f1eff1cfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:09.562286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1044/2098] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:09.569342 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdff0f-5a1e-43c8-bf39-1b4318caca07 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:09.571790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d6cdff0f-5a1e-43c8-bf39-1b4318caca07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:09.611354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:09.649451 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cdff0f-5a1e-43c8-bf39-1b4318caca07 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:09.655126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941571aa-dd0d-4cc4-b312-303a91047052 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:09.657836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941571aa-dd0d-4cc4-b312-303a91047052 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:09.657836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941571aa-dd0d-4cc4-b312-303a91047052 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:09.657836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941571aa-dd0d-4cc4-b312-303a91047052 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:09.670537 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941571aa-dd0d-4cc4-b312-303a91047052 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:09.672209 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941571aa-dd0d-4cc4-b312-303a91047052 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:09.684059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1055/2099] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:19:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:19:09.942309 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee8aff8-f31b-4486-9a72-93f1eff1cfaf None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:09.946632 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1ee8aff8-f31b-4486-9a72-93f1eff1cfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:09.964354 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee8aff8-f31b-4486-9a72-93f1eff1cfaf None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:09.973879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc408cd6-c6e5-4080-9ca9-ef5e808920c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:09.975115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc408cd6-c6e5-4080-9ca9-ef5e808920c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:09.975616 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc408cd6-c6e5-4080-9ca9-ef5e808920c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:09.976669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc408cd6-c6e5-4080-9ca9-ef5e808920c8 admin admin] PATH_INFO: `/v3/projects/42dbb802602d4199a90fd6b7c945ea05` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:09.978805 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc408cd6-c6e5-4080-9ca9-ef5e808920c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42dbb802602d4199a90fd6b7c945ea05)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:09.987426 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc408cd6-c6e5-4080-9ca9-ef5e808920c8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:10.163919 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fc408cd6-c6e5-4080-9ca9-ef5e808920c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42dbb802602d4199a90fd6b7c945ea05', 'request_id': 'req-fc408cd6-c6e5-4080-9ca9-ef5e808920c8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:10.242338 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fc408cd6-c6e5-4080-9ca9-ef5e808920c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42dbb802602d4199a90fd6b7c945ea05', 'request_id': 'req-fc408cd6-c6e5-4080-9ca9-ef5e808920c8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:10.256758 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fc408cd6-c6e5-4080-9ca9-ef5e808920c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42dbb802602d4199a90fd6b7c945ea05', 'request_id': 'req-fc408cd6-c6e5-4080-9ca9-ef5e808920c8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:10.271335 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fc408cd6-c6e5-4080-9ca9-ef5e808920c8 admin admin] The token cache is being invalidate because project 42dbb802602d4199a90fd6b7c945ea05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:19:10.272144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fc408cd6-c6e5-4080-9ca9-ef5e808920c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:10.273561 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fc408cd6-c6e5-4080-9ca9-ef5e808920c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:10.277986 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1045/2100] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:19:09 2022] DELETE /identity/v3/projects/42dbb802602d4199a90fd6b7c945ea05 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:10.280855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a14d5-9d7a-4fb1-8899-8f1451e08cc4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:10.282110 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a14d5-9d7a-4fb1-8899-8f1451e08cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:10.282534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a14d5-9d7a-4fb1-8899-8f1451e08cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:10.305634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6d7a14d5-9d7a-4fb1-8899-8f1451e08cc4 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:10.307198 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6d7a14d5-9d7a-4fb1-8899-8f1451e08cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:10.384368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1056/2101] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:10.392801 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f4ef7-d929-49d0-8592-2ab2d1ecb55c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:10.393118 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f4ef7-d929-49d0-8592-2ab2d1ecb55c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:10.393505 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f4ef7-d929-49d0-8592-2ab2d1ecb55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:10.415463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-842f4ef7-d929-49d0-8592-2ab2d1ecb55c None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:10.419933 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-842f4ef7-d929-49d0-8592-2ab2d1ecb55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:10.430465 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1046/2102] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:10.433857 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a14d5-9d7a-4fb1-8899-8f1451e08cc4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:10.452993 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a14d5-9d7a-4fb1-8899-8f1451e08cc4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:10.459025 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574156f-7dcc-44ce-8718-fb1da4d49066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:10.459853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574156f-7dcc-44ce-8718-fb1da4d49066 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:10.460146 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574156f-7dcc-44ce-8718-fb1da4d49066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:10.460467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b574156f-7dcc-44ce-8718-fb1da4d49066 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:10.461163 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574156f-7dcc-44ce-8718-fb1da4d49066 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:19:10.461649 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574156f-7dcc-44ce-8718-fb1da4d49066 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:10.463448 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b574156f-7dcc-44ce-8718-fb1da4d49066 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:10.478554 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1057/2103] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:19:10.483142 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f4ef7-d929-49d0-8592-2ab2d1ecb55c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:10.527374 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f4ef7-d929-49d0-8592-2ab2d1ecb55c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:10.545814 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d531093-9e3c-471b-b1b4-aa9e638e8f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:10.549730 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d531093-9e3c-471b-b1b4-aa9e638e8f2c admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:10.550287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d531093-9e3c-471b-b1b4-aa9e638e8f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:10.552106 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d531093-9e3c-471b-b1b4-aa9e638e8f2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:10.553018 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d531093-9e3c-471b-b1b4-aa9e638e8f2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:10.556885 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d531093-9e3c-471b-b1b4-aa9e638e8f2c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:10.665778 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1047/2104] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:19:10 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:19:10.670988 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574156f-7dcc-44ce-8718-fb1da4d49066 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:10.687055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b574156f-7dcc-44ce-8718-fb1da4d49066 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:10.806330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e9201-04b0-4b72-8fc0-1182ef552ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:10.807351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e9201-04b0-4b72-8fc0-1182ef552ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:10.807884 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e9201-04b0-4b72-8fc0-1182ef552ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:10.808338 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e9201-04b0-4b72-8fc0-1182ef552ad5 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:10.809997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e9201-04b0-4b72-8fc0-1182ef552ad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:10.813793 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695e9201-04b0-4b72-8fc0-1182ef552ad5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:11.502329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1058/2105] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:19:10 2022] POST /identity/v3/users => generated 291 bytes in 832 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:11.508733 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d531093-9e3c-471b-b1b4-aa9e638e8f2c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:11.525097 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d531093-9e3c-471b-b1b4-aa9e638e8f2c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:11.534994 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c078424-b0ed-4620-8975-d964890b8fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:11.536097 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c078424-b0ed-4620-8975-d964890b8fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:11.536395 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c078424-b0ed-4620-8975-d964890b8fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:11.536670 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c078424-b0ed-4620-8975-d964890b8fb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:11.537434 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c078424-b0ed-4620-8975-d964890b8fb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:11.539120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c078424-b0ed-4620-8975-d964890b8fb4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:11.555110 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1048/2106] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:11.562486 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e9201-04b0-4b72-8fc0-1182ef552ad5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:11.578690 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e9201-04b0-4b72-8fc0-1182ef552ad5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:11.591247 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec5b5c-dd48-4bb1-89d3-d511553ccc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:11.591247 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec5b5c-dd48-4bb1-89d3-d511553ccc77 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:11.591247 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec5b5c-dd48-4bb1-89d3-d511553ccc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:11.591247 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec5b5c-dd48-4bb1-89d3-d511553ccc77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:11.591799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec5b5c-dd48-4bb1-89d3-d511553ccc77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:11.591799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec5b5c-dd48-4bb1-89d3-d511553ccc77 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:11.601847 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1059/2107] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:11.606009 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c078424-b0ed-4620-8975-d964890b8fb4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:11.621321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c078424-b0ed-4620-8975-d964890b8fb4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:11.627787 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a052d2-6aa5-478e-ab18-3bfb87cdf34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:11.628926 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a052d2-6aa5-478e-ab18-3bfb87cdf34c admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:11.629376 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a052d2-6aa5-478e-ab18-3bfb87cdf34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:11.629751 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a052d2-6aa5-478e-ab18-3bfb87cdf34c admin admin] PATH_INFO: `/v3/projects/c711cca7436744a18a8b6a005816b5f8/users/f84d1eed5e17436a88bde34f7b6b16a8/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:11.647951 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a052d2-6aa5-478e-ab18-3bfb87cdf34c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c711cca7436744a18a8b6a005816b5f8, user_id=f84d1eed5e17436a88bde34f7b6b16a8, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:11.651716 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a052d2-6aa5-478e-ab18-3bfb87cdf34c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:11.744006 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1049/2108] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:11 2022] PUT /identity/v3/projects/c711cca7436744a18a8b6a005816b5f8/users/f84d1eed5e17436a88bde34f7b6b16a8/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:11.756010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38685b9-7d6c-4fb7-b259-a28029120a54 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:11.756852 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38685b9-7d6c-4fb7-b259-a28029120a54 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:11.757138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38685b9-7d6c-4fb7-b259-a28029120a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:11.788243 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f38685b9-7d6c-4fb7-b259-a28029120a54 None None] MFA Rules not processed for user `f84d1eed5e17436a88bde34f7b6b16a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:11.790264 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f38685b9-7d6c-4fb7-b259-a28029120a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:11.985676 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b1c17-447b-4e4e-bbc8-763fef962865 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:11.986270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b1c17-447b-4e4e-bbc8-763fef962865 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:11.986753 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b1c17-447b-4e4e-bbc8-763fef962865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:12.092267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1060/2109] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:12.095934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-697b1c17-447b-4e4e-bbc8-763fef962865 None None] MFA Rules not processed for user `ba662104accd44a1aa5e7a473bb51557`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:12.096621 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38685b9-7d6c-4fb7-b259-a28029120a54 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:12.097676 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-697b1c17-447b-4e4e-bbc8-763fef962865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:12.222470 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:12.244333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38685b9-7d6c-4fb7-b259-a28029120a54 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:12.258203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7763a-c656-46e1-93dd-bcb0c8f1bb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:12.259188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7763a-c656-46e1-93dd-bcb0c8f1bb04 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:12.259611 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7763a-c656-46e1-93dd-bcb0c8f1bb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:12.260076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7763a-c656-46e1-93dd-bcb0c8f1bb04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:12.260928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7763a-c656-46e1-93dd-bcb0c8f1bb04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:12.270709 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a7763a-c656-46e1-93dd-bcb0c8f1bb04 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:12.324507 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1050/2110] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:11 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:12.347783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b1c17-447b-4e4e-bbc8-763fef962865 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:12.350741 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-697b1c17-447b-4e4e-bbc8-763fef962865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:12.383224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1061/2111] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:19:12 2022] POST /identity/v3/projects => generated 381 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:19:12.390003 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7763a-c656-46e1-93dd-bcb0c8f1bb04 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:12.483285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b1c17-447b-4e4e-bbc8-763fef962865 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:12.518408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7763a-c656-46e1-93dd-bcb0c8f1bb04 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:12.532497 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc703b34-5eee-47fe-af8d-be97a27955b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:12.533619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc703b34-5eee-47fe-af8d-be97a27955b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:12.534229 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc703b34-5eee-47fe-af8d-be97a27955b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:12.534707 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc703b34-5eee-47fe-af8d-be97a27955b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:12.535855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc703b34-5eee-47fe-af8d-be97a27955b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:12.538355 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc703b34-5eee-47fe-af8d-be97a27955b8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:12.565546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f634f-fbab-49bf-b1dd-aed7af39ac32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:12.567319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f634f-fbab-49bf-b1dd-aed7af39ac32 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:12.567751 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f634f-fbab-49bf-b1dd-aed7af39ac32 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:12.568199 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f634f-fbab-49bf-b1dd-aed7af39ac32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:12.570426 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e63f634f-fbab-49bf-b1dd-aed7af39ac32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:12.654642 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f634f-fbab-49bf-b1dd-aed7af39ac32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:12.656501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1062/2112] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:19:12 2022] POST /identity/v3/users => generated 297 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:12.657305 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f634f-fbab-49bf-b1dd-aed7af39ac32 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:12.657602 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc703b34-5eee-47fe-af8d-be97a27955b8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:12.676524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1051/2113] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:19:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:19:12.678420 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc703b34-5eee-47fe-af8d-be97a27955b8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:12.688601 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac324ea7-f70d-496f-98a5-e718b5b81bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:12.692720 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac324ea7-f70d-496f-98a5-e718b5b81bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:12.692720 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac324ea7-f70d-496f-98a5-e718b5b81bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:12.692720 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac324ea7-f70d-496f-98a5-e718b5b81bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:12.692720 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac324ea7-f70d-496f-98a5-e718b5b81bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:12.695155 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac324ea7-f70d-496f-98a5-e718b5b81bb0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:12.709813 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1063/2114] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:12.711137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f634f-fbab-49bf-b1dd-aed7af39ac32 service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:12.724725 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f634f-fbab-49bf-b1dd-aed7af39ac32 service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:12.731515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbb924-8855-4b1f-b787-4ff83eb1f9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:12.732562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbb924-8855-4b1f-b787-4ff83eb1f9f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:12.732953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbb924-8855-4b1f-b787-4ff83eb1f9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:12.733373 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbbb924-8855-4b1f-b787-4ff83eb1f9f3 admin admin] PATH_INFO: `/v3/projects/5dd4fa4aa93b41fba79da3136cd4dfab/users/9d2a8cb4f8f04b14a1056157542011ee/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:12.759325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbbb924-8855-4b1f-b787-4ff83eb1f9f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dd4fa4aa93b41fba79da3136cd4dfab, user_id=9d2a8cb4f8f04b14a1056157542011ee, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:12.764250 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbbb924-8855-4b1f-b787-4ff83eb1f9f3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:12.883534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1052/2115] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:12 2022] PUT /identity/v3/projects/5dd4fa4aa93b41fba79da3136cd4dfab/users/9d2a8cb4f8f04b14a1056157542011ee/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:12.883693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac324ea7-f70d-496f-98a5-e718b5b81bb0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:12.899155 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac324ea7-f70d-496f-98a5-e718b5b81bb0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:13.050073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902e8ce-4447-4dc5-9362-a5d991cc3363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:13.050962 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902e8ce-4447-4dc5-9362-a5d991cc3363 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:13.051075 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902e8ce-4447-4dc5-9362-a5d991cc3363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:13.051336 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902e8ce-4447-4dc5-9362-a5d991cc3363 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:13.052170 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902e8ce-4447-4dc5-9362-a5d991cc3363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:13.055425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902e8ce-4447-4dc5-9362-a5d991cc3363 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:13.078107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1064/2116] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:19:13.082262 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbb924-8855-4b1f-b787-4ff83eb1f9f3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:13.108707 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbbb924-8855-4b1f-b787-4ff83eb1f9f3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:13.139906 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:13.142729 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73900451-db20-44ce-a09e-e7232ae6edff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:13.143728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73900451-db20-44ce-a09e-e7232ae6edff admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:13.147480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73900451-db20-44ce-a09e-e7232ae6edff admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:13.147480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73900451-db20-44ce-a09e-e7232ae6edff admin admin] PATH_INFO: `/v3/projects/5dd4fa4aa93b41fba79da3136cd4dfab/users/9d2a8cb4f8f04b14a1056157542011ee/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:13.148192 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73900451-db20-44ce-a09e-e7232ae6edff admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dd4fa4aa93b41fba79da3136cd4dfab, user_id=9d2a8cb4f8f04b14a1056157542011ee, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:13.154315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73900451-db20-44ce-a09e-e7232ae6edff admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:13.194151 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1053/2117] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:13 2022] PUT /identity/v3/projects/5dd4fa4aa93b41fba79da3136cd4dfab/users/9d2a8cb4f8f04b14a1056157542011ee/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:13.199494 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e1d34-d77f-4cd1-b648-14e2e91b6989 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:13.202207 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e1d34-d77f-4cd1-b648-14e2e91b6989 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:13.202570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e1d34-d77f-4cd1-b648-14e2e91b6989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:13.268549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-500e1d34-d77f-4cd1-b648-14e2e91b6989 None None] MFA Rules not processed for user `9d2a8cb4f8f04b14a1056157542011ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:13.270843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-500e1d34-d77f-4cd1-b648-14e2e91b6989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:13.439718 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:13.494904 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1065/2118] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:13.506626 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73900451-db20-44ce-a09e-e7232ae6edff admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:13.508558 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-73900451-db20-44ce-a09e-e7232ae6edff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:13.612417 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73900451-db20-44ce-a09e-e7232ae6edff admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:13.618145 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3fc9f7-b62a-481a-b314-50f0356288e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:13.619541 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3fc9f7-b62a-481a-b314-50f0356288e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:13.619987 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3fc9f7-b62a-481a-b314-50f0356288e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:13.621711 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3fc9f7-b62a-481a-b314-50f0356288e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:13.636074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3fc9f7-b62a-481a-b314-50f0356288e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:13.639168 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3fc9f7-b62a-481a-b314-50f0356288e6 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:13.652818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1054/2119] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:19:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:19:14.494795 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e1d34-d77f-4cd1-b648-14e2e91b6989 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:14.508235 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e1d34-d77f-4cd1-b648-14e2e91b6989 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:14.528934 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49d5c3-9ebd-4edd-832e-24976933e28b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:14.528934 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49d5c3-9ebd-4edd-832e-24976933e28b service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:14.528934 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49d5c3-9ebd-4edd-832e-24976933e28b service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:14.528934 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49d5c3-9ebd-4edd-832e-24976933e28b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:14.656231 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49d5c3-9ebd-4edd-832e-24976933e28b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:14.658151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49d5c3-9ebd-4edd-832e-24976933e28b service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:14.676583 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1066/2120] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:19:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:19:16.051467 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3fc9f7-b62a-481a-b314-50f0356288e6 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:16.054821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1e3fc9f7-b62a-481a-b314-50f0356288e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:16.256686 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3fc9f7-b62a-481a-b314-50f0356288e6 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:16.264461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b8f12-903a-45f4-a5df-497815a97601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:16.268722 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b8f12-903a-45f4-a5df-497815a97601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:16.268722 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b8f12-903a-45f4-a5df-497815a97601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:16.268722 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b8f12-903a-45f4-a5df-497815a97601 admin admin] PATH_INFO: `/v3/users/e1563be888b74ee789955eb285cd7376` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:16.269673 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b8f12-903a-45f4-a5df-497815a97601 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1563be888b74ee789955eb285cd7376)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:16.274660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b8f12-903a-45f4-a5df-497815a97601 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:16.510732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b56b8f12-903a-45f4-a5df-497815a97601 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1563be888b74ee789955eb285cd7376'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:16.532011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b56b8f12-903a-45f4-a5df-497815a97601 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1563be888b74ee789955eb285cd7376'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:16.629906 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b56b8f12-903a-45f4-a5df-497815a97601 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1563be888b74ee789955eb285cd7376'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:16.630909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b56b8f12-903a-45f4-a5df-497815a97601 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1563be888b74ee789955eb285cd7376'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:16.632707 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b56b8f12-903a-45f4-a5df-497815a97601 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1563be888b74ee789955eb285cd7376'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:16.675958 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1055/2121] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:19:16 2022] DELETE /identity/v3/users/e1563be888b74ee789955eb285cd7376 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:16.680663 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ee0fb-f87d-4350-8832-fbc0d2fd4d50 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:16.681117 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ee0fb-f87d-4350-8832-fbc0d2fd4d50 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:16.681236 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ee0fb-f87d-4350-8832-fbc0d2fd4d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:16.703289 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ed4ee0fb-f87d-4350-8832-fbc0d2fd4d50 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:16.704815 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ed4ee0fb-f87d-4350-8832-fbc0d2fd4d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:16.737063 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:16.827776 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1067/2122] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:16.833610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:16.848664 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b8f12-903a-45f4-a5df-497815a97601 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:16.852265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b56b8f12-903a-45f4-a5df-497815a97601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:16.968849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b8f12-903a-45f4-a5df-497815a97601 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:16.974085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e9333-745f-4097-ad88-0333fe4c86f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:16.974877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e9333-745f-4097-ad88-0333fe4c86f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:16.975163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e9333-745f-4097-ad88-0333fe4c86f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:16.975611 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e9333-745f-4097-ad88-0333fe4c86f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:16.988888 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e9333-745f-4097-ad88-0333fe4c86f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:16.992326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e9333-745f-4097-ad88-0333fe4c86f4 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:17.018934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1056/2123] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:19:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:19:17.227298 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb593aff-d1b3-495c-9dee-c434094df902 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:17.227298 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb593aff-d1b3-495c-9dee-c434094df902 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:17.233192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb593aff-d1b3-495c-9dee-c434094df902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:17.267371 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-eb593aff-d1b3-495c-9dee-c434094df902 None None] MFA Rules not processed for user `f3821e46c09b449d85c46c58d99bf607`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:17.268617 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-eb593aff-d1b3-495c-9dee-c434094df902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:17.382679 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1068/2124] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:17.907216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e9333-745f-4097-ad88-0333fe4c86f4 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:17.909619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-468e9333-745f-4097-ad88-0333fe4c86f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:17.933814 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e9333-745f-4097-ad88-0333fe4c86f4 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:17.948662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdfe689-4338-47da-9801-a12b21ac6278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:17.949562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdfe689-4338-47da-9801-a12b21ac6278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:17.950629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdfe689-4338-47da-9801-a12b21ac6278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:17.950952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdfe689-4338-47da-9801-a12b21ac6278 admin admin] PATH_INFO: `/v3/projects/b33acd01b27b41208c338588719e0acf` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:17.958292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdfe689-4338-47da-9801-a12b21ac6278 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b33acd01b27b41208c338588719e0acf)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:17.958292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdfe689-4338-47da-9801-a12b21ac6278 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:18.171176 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bcdfe689-4338-47da-9801-a12b21ac6278 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b33acd01b27b41208c338588719e0acf', 'request_id': 'req-bcdfe689-4338-47da-9801-a12b21ac6278'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:18.313389 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bcdfe689-4338-47da-9801-a12b21ac6278 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b33acd01b27b41208c338588719e0acf', 'request_id': 'req-bcdfe689-4338-47da-9801-a12b21ac6278'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:18.331228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bcdfe689-4338-47da-9801-a12b21ac6278 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b33acd01b27b41208c338588719e0acf', 'request_id': 'req-bcdfe689-4338-47da-9801-a12b21ac6278'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:18.360471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bcdfe689-4338-47da-9801-a12b21ac6278 admin admin] The token cache is being invalidate because project b33acd01b27b41208c338588719e0acf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:19:18.361039 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bcdfe689-4338-47da-9801-a12b21ac6278 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:18.362322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-bcdfe689-4338-47da-9801-a12b21ac6278 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:18.369294 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1057/2125] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:19:17 2022] DELETE /identity/v3/projects/b33acd01b27b41208c338588719e0acf => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:18.373450 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb593aff-d1b3-495c-9dee-c434094df902 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:18.385192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb593aff-d1b3-495c-9dee-c434094df902 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:18.479371 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c137f-eab5-4537-9cac-54ddb0ceb479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:18.480531 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c137f-eab5-4537-9cac-54ddb0ceb479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:18.480818 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c137f-eab5-4537-9cac-54ddb0ceb479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:18.481268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c137f-eab5-4537-9cac-54ddb0ceb479 admin admin] PATH_INFO: `/v3/users/c2ddbf73041546fa92a55b5889473812` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:18.482754 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c137f-eab5-4537-9cac-54ddb0ceb479 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2ddbf73041546fa92a55b5889473812)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:18.484931 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c137f-eab5-4537-9cac-54ddb0ceb479 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:18.587953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:18.732844 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f51c137f-eab5-4537-9cac-54ddb0ceb479 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2ddbf73041546fa92a55b5889473812'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:18.746464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f51c137f-eab5-4537-9cac-54ddb0ceb479 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2ddbf73041546fa92a55b5889473812'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:18.747344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f51c137f-eab5-4537-9cac-54ddb0ceb479 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2ddbf73041546fa92a55b5889473812'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:18.748048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f51c137f-eab5-4537-9cac-54ddb0ceb479 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2ddbf73041546fa92a55b5889473812'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:18.770415 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f51c137f-eab5-4537-9cac-54ddb0ceb479 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2ddbf73041546fa92a55b5889473812'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:18.903080 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1069/2126] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:19:18 2022] DELETE /identity/v3/users/c2ddbf73041546fa92a55b5889473812 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:18.973287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38f1cf-baef-484b-877a-253e599c876b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:18.973838 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38f1cf-baef-484b-877a-253e599c876b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:18.973991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38f1cf-baef-484b-877a-253e599c876b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:18.996041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c137f-eab5-4537-9cac-54ddb0ceb479 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:18.996041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f51c137f-eab5-4537-9cac-54ddb0ceb479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:19.051480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7d38f1cf-baef-484b-877a-253e599c876b None None] MFA Rules not processed for user `4fb36d1e057543a3abcf6d6c2564d9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:19.052736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7d38f1cf-baef-484b-877a-253e599c876b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:19.172813 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c137f-eab5-4537-9cac-54ddb0ceb479 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:19.188841 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a928268-55f3-4472-8970-361377fb9976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:19.189665 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a928268-55f3-4472-8970-361377fb9976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:19.189960 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a928268-55f3-4472-8970-361377fb9976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:19.190243 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a928268-55f3-4472-8970-361377fb9976 admin admin] PATH_INFO: `/v3/users/f84d1eed5e17436a88bde34f7b6b16a8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:19.193157 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a928268-55f3-4472-8970-361377fb9976 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f84d1eed5e17436a88bde34f7b6b16a8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:19.198498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a928268-55f3-4472-8970-361377fb9976 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:19.235529 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1058/2127] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:19.240884 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38f1cf-baef-484b-877a-253e599c876b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:19.242270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7d38f1cf-baef-484b-877a-253e599c876b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:19.256940 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38f1cf-baef-484b-877a-253e599c876b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:19.267807 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a2e25-657c-4077-b55b-d86dc2bdcc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:19.267807 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a2e25-657c-4077-b55b-d86dc2bdcc87 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:19.267807 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a2e25-657c-4077-b55b-d86dc2bdcc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:19.267807 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a2e25-657c-4077-b55b-d86dc2bdcc87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:19.268201 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a2e25-657c-4077-b55b-d86dc2bdcc87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:19.268201 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a2e25-657c-4077-b55b-d86dc2bdcc87 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:19.395289 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1059/2128] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:19:19 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:19:19.399587 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a2e25-657c-4077-b55b-d86dc2bdcc87 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:19.479640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a2e25-657c-4077-b55b-d86dc2bdcc87 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:19.486325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590548f8-9c6b-40b1-bfe1-17c9b3753830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:19.487136 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590548f8-9c6b-40b1-bfe1-17c9b3753830 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:19.487608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590548f8-9c6b-40b1-bfe1-17c9b3753830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:19.488044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590548f8-9c6b-40b1-bfe1-17c9b3753830 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:19.488899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590548f8-9c6b-40b1-bfe1-17c9b3753830 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:19.491499 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590548f8-9c6b-40b1-bfe1-17c9b3753830 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:19.495716 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6a928268-55f3-4472-8970-361377fb9976 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f84d1eed5e17436a88bde34f7b6b16a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:19.510866 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6a928268-55f3-4472-8970-361377fb9976 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f84d1eed5e17436a88bde34f7b6b16a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:19.511711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6a928268-55f3-4472-8970-361377fb9976 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f84d1eed5e17436a88bde34f7b6b16a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:19.512942 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6a928268-55f3-4472-8970-361377fb9976 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f84d1eed5e17436a88bde34f7b6b16a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:19.521226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6a928268-55f3-4472-8970-361377fb9976 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f84d1eed5e17436a88bde34f7b6b16a8'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:19.639115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1060/2129] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:19:19 2022] POST /identity/v3/users => generated 299 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:19:19.641127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590548f8-9c6b-40b1-bfe1-17c9b3753830 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:19.642953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-590548f8-9c6b-40b1-bfe1-17c9b3753830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:19.664932 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590548f8-9c6b-40b1-bfe1-17c9b3753830 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:19.676233 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1070/2130] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:19:18 2022] DELETE /identity/v3/users/f84d1eed5e17436a88bde34f7b6b16a8 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:19.680753 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91333a4b-a629-4e60-8cab-3a18b6ebd417 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:19.681040 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91333a4b-a629-4e60-8cab-3a18b6ebd417 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:19.681510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91333a4b-a629-4e60-8cab-3a18b6ebd417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:19.713836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-91333a4b-a629-4e60-8cab-3a18b6ebd417 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:19.715394 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-91333a4b-a629-4e60-8cab-3a18b6ebd417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:19.746906 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d2fba-e770-4fbb-acb4-ef8bb6e63e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:19.752087 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d2fba-e770-4fbb-acb4-ef8bb6e63e74 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:19.752087 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d2fba-e770-4fbb-acb4-ef8bb6e63e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:19.752087 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d2fba-e770-4fbb-acb4-ef8bb6e63e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:19.752087 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d2fba-e770-4fbb-acb4-ef8bb6e63e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:19.752087 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d2fba-e770-4fbb-acb4-ef8bb6e63e74 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:19.764047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1061/2131] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:19:19.768070 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d2fba-e770-4fbb-acb4-ef8bb6e63e74 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:19.780690 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d2fba-e770-4fbb-acb4-ef8bb6e63e74 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:19.783358 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1071/2132] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:19.784808 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504d7e2-458a-4745-99e7-41ea36302bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:19.785696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504d7e2-458a-4745-99e7-41ea36302bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:19.785984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504d7e2-458a-4745-99e7-41ea36302bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:19.786519 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504d7e2-458a-4745-99e7-41ea36302bc0 admin admin] PATH_INFO: `/v3/projects/d308801b882b46209e83593e0607ee89/users/3ec36b2f17324fe699593c3aebb6c39c/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:19.793219 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91333a4b-a629-4e60-8cab-3a18b6ebd417 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:19.794871 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-91333a4b-a629-4e60-8cab-3a18b6ebd417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:19.799371 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:19.816282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1504d7e2-458a-4745-99e7-41ea36302bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d308801b882b46209e83593e0607ee89, user_id=3ec36b2f17324fe699593c3aebb6c39c, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:19.816282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1504d7e2-458a-4745-99e7-41ea36302bc0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:19.864688 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91333a4b-a629-4e60-8cab-3a18b6ebd417 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:19.869833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f4361-ba4a-478c-9566-64ea26fc9a89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:19.870582 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f4361-ba4a-478c-9566-64ea26fc9a89 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:19.870913 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f4361-ba4a-478c-9566-64ea26fc9a89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:19.871208 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f4361-ba4a-478c-9566-64ea26fc9a89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:19.889643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f4361-ba4a-478c-9566-64ea26fc9a89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:19.891711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f4361-ba4a-478c-9566-64ea26fc9a89 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:19.901463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1062/2133] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:19 2022] PUT /identity/v3/projects/d308801b882b46209e83593e0607ee89/users/3ec36b2f17324fe699593c3aebb6c39c/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:19:19.906018 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504d7e2-458a-4745-99e7-41ea36302bc0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:19.907781 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1504d7e2-458a-4745-99e7-41ea36302bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:19.908209 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1072/2134] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:19:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:19:19.909094 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f4361-ba4a-478c-9566-64ea26fc9a89 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:19.922336 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f4361-ba4a-478c-9566-64ea26fc9a89 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:20.007220 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504d7e2-458a-4745-99e7-41ea36302bc0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:20.015174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690470e-3770-4019-baba-6c45ebdf0941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:20.022916 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690470e-3770-4019-baba-6c45ebdf0941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:20.022916 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690470e-3770-4019-baba-6c45ebdf0941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:20.022916 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3690470e-3770-4019-baba-6c45ebdf0941 admin admin] PATH_INFO: `/v3/projects/ca84616386aa40a78c20e0613e31997c` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:20.022916 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690470e-3770-4019-baba-6c45ebdf0941 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca84616386aa40a78c20e0613e31997c)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:20.022916 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3690470e-3770-4019-baba-6c45ebdf0941 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:20.061917 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e364f-b05f-4bd1-aa07-b75425e49ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:20.062668 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e364f-b05f-4bd1-aa07-b75425e49ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:20.063055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e364f-b05f-4bd1-aa07-b75425e49ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:20.063241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e364f-b05f-4bd1-aa07-b75425e49ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:20.064043 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e364f-b05f-4bd1-aa07-b75425e49ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:20.068080 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e364f-b05f-4bd1-aa07-b75425e49ce0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:20.097502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1073/2135] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:20.097502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e364f-b05f-4bd1-aa07-b75425e49ce0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:20.104584 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e364f-b05f-4bd1-aa07-b75425e49ce0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:20.110727 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ba2dc-8bb7-4186-bd17-700e574aa1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:20.111900 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ba2dc-8bb7-4186-bd17-700e574aa1bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:20.112287 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ba2dc-8bb7-4186-bd17-700e574aa1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:20.112770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ba2dc-8bb7-4186-bd17-700e574aa1bb admin admin] PATH_INFO: `/v3/projects/d308801b882b46209e83593e0607ee89/users/3ec36b2f17324fe699593c3aebb6c39c/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:20.114951 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ba2dc-8bb7-4186-bd17-700e574aa1bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d308801b882b46209e83593e0607ee89, user_id=3ec36b2f17324fe699593c3aebb6c39c, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:20.117795 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ba2dc-8bb7-4186-bd17-700e574aa1bb admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:20.137205 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1074/2136] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:20 2022] PUT /identity/v3/projects/d308801b882b46209e83593e0607ee89/users/3ec36b2f17324fe699593c3aebb6c39c/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:20.216092 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3690470e-3770-4019-baba-6c45ebdf0941 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca84616386aa40a78c20e0613e31997c', 'request_id': 'req-3690470e-3770-4019-baba-6c45ebdf0941'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:20.310326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3690470e-3770-4019-baba-6c45ebdf0941 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca84616386aa40a78c20e0613e31997c', 'request_id': 'req-3690470e-3770-4019-baba-6c45ebdf0941'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:20.321341 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3690470e-3770-4019-baba-6c45ebdf0941 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca84616386aa40a78c20e0613e31997c', 'request_id': 'req-3690470e-3770-4019-baba-6c45ebdf0941'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:20.345438 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3690470e-3770-4019-baba-6c45ebdf0941 admin admin] The token cache is being invalidate because project ca84616386aa40a78c20e0613e31997c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:19:20.347592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3690470e-3770-4019-baba-6c45ebdf0941 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:20.349584 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3690470e-3770-4019-baba-6c45ebdf0941 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:20.351922 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1063/2137] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:19:19 2022] DELETE /identity/v3/projects/ca84616386aa40a78c20e0613e31997c => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:20.356692 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ba2dc-8bb7-4186-bd17-700e574aa1bb admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:20.358970 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2b8ba2dc-8bb7-4186-bd17-700e574aa1bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:20.467310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ba2dc-8bb7-4186-bd17-700e574aa1bb admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:20.540227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:20.548801 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac55abe-f4f6-4b4a-b9b8-b30dca8e6855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:20.548801 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac55abe-f4f6-4b4a-b9b8-b30dca8e6855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:20.548801 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac55abe-f4f6-4b4a-b9b8-b30dca8e6855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:20.548801 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac55abe-f4f6-4b4a-b9b8-b30dca8e6855 admin admin] PATH_INFO: `/v3/users/b26cede92240458aa227a9b30b26f538` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:20.549625 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac55abe-f4f6-4b4a-b9b8-b30dca8e6855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b26cede92240458aa227a9b30b26f538)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:20.553276 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac55abe-f4f6-4b4a-b9b8-b30dca8e6855 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:20.662721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:20.712707 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690470e-3770-4019-baba-6c45ebdf0941 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:20.714418 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3690470e-3770-4019-baba-6c45ebdf0941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:20.739134 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3690470e-3770-4019-baba-6c45ebdf0941 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:20.759796 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bde132-9dac-4126-8aaa-4481f53b52bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:20.779146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bde132-9dac-4126-8aaa-4481f53b52bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:20.779146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bde132-9dac-4126-8aaa-4481f53b52bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:20.779146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bde132-9dac-4126-8aaa-4481f53b52bf admin admin] PATH_INFO: `/v3/projects/c711cca7436744a18a8b6a005816b5f8` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:20.779146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bde132-9dac-4126-8aaa-4481f53b52bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c711cca7436744a18a8b6a005816b5f8)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:20.779146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bde132-9dac-4126-8aaa-4481f53b52bf admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:20.847824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-bac55abe-f4f6-4b4a-b9b8-b30dca8e6855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b26cede92240458aa227a9b30b26f538'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:20.873026 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-bac55abe-f4f6-4b4a-b9b8-b30dca8e6855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b26cede92240458aa227a9b30b26f538'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:20.874531 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-bac55abe-f4f6-4b4a-b9b8-b30dca8e6855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b26cede92240458aa227a9b30b26f538'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:20.875630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-bac55abe-f4f6-4b4a-b9b8-b30dca8e6855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b26cede92240458aa227a9b30b26f538'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:20.888526 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-bac55abe-f4f6-4b4a-b9b8-b30dca8e6855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b26cede92240458aa227a9b30b26f538'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:21.007490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-79bde132-9dac-4126-8aaa-4481f53b52bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c711cca7436744a18a8b6a005816b5f8', 'request_id': 'req-79bde132-9dac-4126-8aaa-4481f53b52bf'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:21.032195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1075/2138] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:19:20 2022] DELETE /identity/v3/users/b26cede92240458aa227a9b30b26f538 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:21.134058 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-79bde132-9dac-4126-8aaa-4481f53b52bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c711cca7436744a18a8b6a005816b5f8', 'request_id': 'req-79bde132-9dac-4126-8aaa-4481f53b52bf'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:21.150927 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-79bde132-9dac-4126-8aaa-4481f53b52bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c711cca7436744a18a8b6a005816b5f8', 'request_id': 'req-79bde132-9dac-4126-8aaa-4481f53b52bf'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:21.187039 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-79bde132-9dac-4126-8aaa-4481f53b52bf admin admin] The token cache is being invalidate because project c711cca7436744a18a8b6a005816b5f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:19:21.187342 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-79bde132-9dac-4126-8aaa-4481f53b52bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:21.188039 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-79bde132-9dac-4126-8aaa-4481f53b52bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:21.189631 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1064/2139] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:19:20 2022] DELETE /identity/v3/projects/c711cca7436744a18a8b6a005816b5f8 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:21.193819 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac55abe-f4f6-4b4a-b9b8-b30dca8e6855 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:21.195419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-bac55abe-f4f6-4b4a-b9b8-b30dca8e6855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:21.328576 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac55abe-f4f6-4b4a-b9b8-b30dca8e6855 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:21.336803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568d1b3-fd36-4963-b808-7f4537d48587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:21.338733 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568d1b3-fd36-4963-b808-7f4537d48587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:21.344111 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568d1b3-fd36-4963-b808-7f4537d48587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:21.344111 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568d1b3-fd36-4963-b808-7f4537d48587 admin admin] PATH_INFO: `/v3/users/9d2a8cb4f8f04b14a1056157542011ee` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:21.344111 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568d1b3-fd36-4963-b808-7f4537d48587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d2a8cb4f8f04b14a1056157542011ee)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:21.346700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568d1b3-fd36-4963-b808-7f4537d48587 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:21.576263 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8568d1b3-fd36-4963-b808-7f4537d48587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d2a8cb4f8f04b14a1056157542011ee'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:21.592863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8568d1b3-fd36-4963-b808-7f4537d48587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d2a8cb4f8f04b14a1056157542011ee'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:21.594917 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8568d1b3-fd36-4963-b808-7f4537d48587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d2a8cb4f8f04b14a1056157542011ee'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:21.595755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8568d1b3-fd36-4963-b808-7f4537d48587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d2a8cb4f8f04b14a1056157542011ee'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:21.602926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-8568d1b3-fd36-4963-b808-7f4537d48587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d2a8cb4f8f04b14a1056157542011ee'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:21.629326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bde132-9dac-4126-8aaa-4481f53b52bf admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:21.629326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-79bde132-9dac-4126-8aaa-4481f53b52bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:21.707407 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1076/2140] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:19:21 2022] DELETE /identity/v3/users/9d2a8cb4f8f04b14a1056157542011ee => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:21.730387 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bde132-9dac-4126-8aaa-4481f53b52bf admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:21.883496 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f64fdb8-f1e8-4640-80da-255eda0a6859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:21.885048 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64fdb8-f1e8-4640-80da-255eda0a6859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:21.885591 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64fdb8-f1e8-4640-80da-255eda0a6859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:21.885879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f64fdb8-f1e8-4640-80da-255eda0a6859 admin admin] PATH_INFO: `/v3/projects/daf93111b68d48c490ed1c1d733fc3a1` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:21.888610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f64fdb8-f1e8-4640-80da-255eda0a6859 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daf93111b68d48c490ed1c1d733fc3a1)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:21.891087 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f64fdb8-f1e8-4640-80da-255eda0a6859 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:22.067230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3f64fdb8-f1e8-4640-80da-255eda0a6859 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daf93111b68d48c490ed1c1d733fc3a1', 'request_id': 'req-3f64fdb8-f1e8-4640-80da-255eda0a6859'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:22.187495 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3f64fdb8-f1e8-4640-80da-255eda0a6859 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daf93111b68d48c490ed1c1d733fc3a1', 'request_id': 'req-3f64fdb8-f1e8-4640-80da-255eda0a6859'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:22.199173 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3f64fdb8-f1e8-4640-80da-255eda0a6859 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daf93111b68d48c490ed1c1d733fc3a1', 'request_id': 'req-3f64fdb8-f1e8-4640-80da-255eda0a6859'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:22.229966 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3f64fdb8-f1e8-4640-80da-255eda0a6859 admin admin] The token cache is being invalidate because project daf93111b68d48c490ed1c1d733fc3a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:19:22.230677 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3f64fdb8-f1e8-4640-80da-255eda0a6859 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:22.232149 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3f64fdb8-f1e8-4640-80da-255eda0a6859 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:22.239087 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1065/2141] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:19:21 2022] DELETE /identity/v3/projects/daf93111b68d48c490ed1c1d733fc3a1 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:22.241287 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48776a-b608-48fc-865d-9f51bfe37491 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:22.241424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48776a-b608-48fc-865d-9f51bfe37491 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:22.241607 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48776a-b608-48fc-865d-9f51bfe37491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:22.302147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ab48776a-b608-48fc-865d-9f51bfe37491 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:22.304744 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ab48776a-b608-48fc-865d-9f51bfe37491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:22.471213 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1077/2142] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:22.479581 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4e0de-8bd1-454c-96ae-fbe5fb3244d6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:22.480556 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4e0de-8bd1-454c-96ae-fbe5fb3244d6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:22.484023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4e0de-8bd1-454c-96ae-fbe5fb3244d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:22.503479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48776a-b608-48fc-865d-9f51bfe37491 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:22.508319 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:22.513867 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ab48776a-b608-48fc-865d-9f51bfe37491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:22.535793 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-acb4e0de-8bd1-454c-96ae-fbe5fb3244d6 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:22.537064 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-acb4e0de-8bd1-454c-96ae-fbe5fb3244d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:22.541105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48776a-b608-48fc-865d-9f51bfe37491 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:22.558583 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff0ebc-04eb-40c1-b07c-021549b29123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:22.561265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff0ebc-04eb-40c1-b07c-021549b29123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:22.561555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff0ebc-04eb-40c1-b07c-021549b29123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:22.562877 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff0ebc-04eb-40c1-b07c-021549b29123 admin admin] PATH_INFO: `/v3/projects/5dd4fa4aa93b41fba79da3136cd4dfab` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:22.566757 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ff0ebc-04eb-40c1-b07c-021549b29123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dd4fa4aa93b41fba79da3136cd4dfab)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:22.567022 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1066/2143] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:22.570448 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4e0de-8bd1-454c-96ae-fbe5fb3244d6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:22.574361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ff0ebc-04eb-40c1-b07c-021549b29123 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:22.606951 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4e0de-8bd1-454c-96ae-fbe5fb3244d6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:22.621117 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861e668-cb21-421e-92e3-e1f351cd09e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:22.622101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861e668-cb21-421e-92e3-e1f351cd09e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:22.622419 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861e668-cb21-421e-92e3-e1f351cd09e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:22.622700 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861e668-cb21-421e-92e3-e1f351cd09e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:22.623877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861e668-cb21-421e-92e3-e1f351cd09e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:19:22.624437 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861e668-cb21-421e-92e3-e1f351cd09e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:22.626526 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861e668-cb21-421e-92e3-e1f351cd09e0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:22.661886 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1067/2144] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:19:22.666486 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861e668-cb21-421e-92e3-e1f351cd09e0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:22.679617 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861e668-cb21-421e-92e3-e1f351cd09e0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:22.687191 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94e9ae-a850-4947-a790-37322588f876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:22.688306 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94e9ae-a850-4947-a790-37322588f876 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:22.689928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94e9ae-a850-4947-a790-37322588f876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:22.690570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e94e9ae-a850-4947-a790-37322588f876 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:22.691741 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94e9ae-a850-4947-a790-37322588f876 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:22.694657 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e94e9ae-a850-4947-a790-37322588f876 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:22.828849 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c9ff0ebc-04eb-40c1-b07c-021549b29123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dd4fa4aa93b41fba79da3136cd4dfab', 'request_id': 'req-c9ff0ebc-04eb-40c1-b07c-021549b29123'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:22.837641 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1068/2145] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:19:22 2022] POST /identity/v3/projects => generated 365 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:22.842560 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94e9ae-a850-4947-a790-37322588f876 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:22.850709 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c9ff0ebc-04eb-40c1-b07c-021549b29123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dd4fa4aa93b41fba79da3136cd4dfab', 'request_id': 'req-c9ff0ebc-04eb-40c1-b07c-021549b29123'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:22.863265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c9ff0ebc-04eb-40c1-b07c-021549b29123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dd4fa4aa93b41fba79da3136cd4dfab', 'request_id': 'req-c9ff0ebc-04eb-40c1-b07c-021549b29123'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:22.928755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e94e9ae-a850-4947-a790-37322588f876 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:22.935186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc656d1-a2b5-4bf0-b03a-f9c7b02981c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:22.936263 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc656d1-a2b5-4bf0-b03a-f9c7b02981c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:22.936553 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc656d1-a2b5-4bf0-b03a-f9c7b02981c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:22.936866 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc656d1-a2b5-4bf0-b03a-f9c7b02981c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:22.937617 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc656d1-a2b5-4bf0-b03a-f9c7b02981c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:22.943038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc656d1-a2b5-4bf0-b03a-f9c7b02981c3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:23.011687 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c9ff0ebc-04eb-40c1-b07c-021549b29123 admin admin] The token cache is being invalidate because project 5dd4fa4aa93b41fba79da3136cd4dfab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:19:23.012175 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c9ff0ebc-04eb-40c1-b07c-021549b29123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:23.013954 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-c9ff0ebc-04eb-40c1-b07c-021549b29123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:23.019340 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1078/2146] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:19:22 2022] DELETE /identity/v3/projects/5dd4fa4aa93b41fba79da3136cd4dfab => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:23.027053 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92a9f3-2711-472e-b10b-4cef975a319d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:23.028280 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92a9f3-2711-472e-b10b-4cef975a319d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:23.029021 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92a9f3-2711-472e-b10b-4cef975a319d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:23.071451 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1069/2147] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:19:22 2022] POST /identity/v3/users => generated 283 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:23.075664 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc656d1-a2b5-4bf0-b03a-f9c7b02981c3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:23.078010 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ccc656d1-a2b5-4bf0-b03a-f9c7b02981c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:23.083576 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ea92a9f3-2711-472e-b10b-4cef975a319d None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:23.086690 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ea92a9f3-2711-472e-b10b-4cef975a319d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:23.103542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1079/2148] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:23.112238 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc656d1-a2b5-4bf0-b03a-f9c7b02981c3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:23.116265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a597b06-b8ad-426b-b0ff-ade6f4108425 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:23.117013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a597b06-b8ad-426b-b0ff-ade6f4108425 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:23.117520 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a597b06-b8ad-426b-b0ff-ade6f4108425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:23.127127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d29813-f519-4dea-8551-08a00e7a1712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:23.130775 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d29813-f519-4dea-8551-08a00e7a1712 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:23.131241 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d29813-f519-4dea-8551-08a00e7a1712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:23.131677 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d29813-f519-4dea-8551-08a00e7a1712 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:23.132616 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d29813-f519-4dea-8551-08a00e7a1712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:23.136908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d29813-f519-4dea-8551-08a00e7a1712 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:23.200435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1070/2149] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:19:23.201748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-0a597b06-b8ad-426b-b0ff-ade6f4108425 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:23.201748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0a597b06-b8ad-426b-b0ff-ade6f4108425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:23.202004 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d29813-f519-4dea-8551-08a00e7a1712 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:23.202093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1080/2150] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:23.202093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a597b06-b8ad-426b-b0ff-ade6f4108425 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:23.202263 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d29813-f519-4dea-8551-08a00e7a1712 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:23.206750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a597b06-b8ad-426b-b0ff-ade6f4108425 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:23.210270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165bb54-890e-4f7e-830c-eeb5489c9c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:23.211145 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165bb54-890e-4f7e-830c-eeb5489c9c97 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:23.211600 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165bb54-890e-4f7e-830c-eeb5489c9c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:23.212050 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165bb54-890e-4f7e-830c-eeb5489c9c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:23.213139 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165bb54-890e-4f7e-830c-eeb5489c9c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:23.213923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d48be8-9f75-453f-b4e8-0b8a16d6f516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:23.214739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d48be8-9f75-453f-b4e8-0b8a16d6f516 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:23.215152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d48be8-9f75-453f-b4e8-0b8a16d6f516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:23.215620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d48be8-9f75-453f-b4e8-0b8a16d6f516 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:23.216609 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d48be8-9f75-453f-b4e8-0b8a16d6f516 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:19:23.217100 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d48be8-9f75-453f-b4e8-0b8a16d6f516 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:23.219059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d48be8-9f75-453f-b4e8-0b8a16d6f516 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:23.221160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165bb54-890e-4f7e-830c-eeb5489c9c97 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:23.243218 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1081/2151] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:19:23.250351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1071/2152] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:23.252064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d48be8-9f75-453f-b4e8-0b8a16d6f516 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:23.254116 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165bb54-890e-4f7e-830c-eeb5489c9c97 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:23.272922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d48be8-9f75-453f-b4e8-0b8a16d6f516 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:23.284357 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93f509-4959-4293-8ab7-e25112a7a5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:23.285204 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93f509-4959-4293-8ab7-e25112a7a5a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:23.285490 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93f509-4959-4293-8ab7-e25112a7a5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:23.285769 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93f509-4959-4293-8ab7-e25112a7a5a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:23.286535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93f509-4959-4293-8ab7-e25112a7a5a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:23.287166 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165bb54-890e-4f7e-830c-eeb5489c9c97 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:23.293031 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696c1b5-1b24-41a8-9cfd-243f579bd3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:23.295585 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696c1b5-1b24-41a8-9cfd-243f579bd3d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:23.296113 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93f509-4959-4293-8ab7-e25112a7a5a6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:23.296510 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696c1b5-1b24-41a8-9cfd-243f579bd3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:23.296961 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696c1b5-1b24-41a8-9cfd-243f579bd3d8 admin admin] PATH_INFO: `/v3/projects/48a2b0b627024d2a9d3309b4e5e4bbf5/users/b45a74c5a7d145bc8371ada44fbf9916/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:23.316213 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696c1b5-1b24-41a8-9cfd-243f579bd3d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48a2b0b627024d2a9d3309b4e5e4bbf5, user_id=b45a74c5a7d145bc8371ada44fbf9916, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:23.319704 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696c1b5-1b24-41a8-9cfd-243f579bd3d8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:23.422296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1072/2153] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:23 2022] PUT /identity/v3/projects/48a2b0b627024d2a9d3309b4e5e4bbf5/users/b45a74c5a7d145bc8371ada44fbf9916/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:23.435806 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560bcce-66c7-4d48-b008-14ad18a97d2f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:23.436539 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560bcce-66c7-4d48-b008-14ad18a97d2f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:23.437025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560bcce-66c7-4d48-b008-14ad18a97d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:23.449004 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1082/2154] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:19:23 2022] POST /identity/v3/projects => generated 387 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:23.456796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93f509-4959-4293-8ab7-e25112a7a5a6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:23.507536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8560bcce-66c7-4d48-b008-14ad18a97d2f None None] MFA Rules not processed for user `b45a74c5a7d145bc8371ada44fbf9916`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:23.513272 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8560bcce-66c7-4d48-b008-14ad18a97d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:23.530615 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:23.600703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93f509-4959-4293-8ab7-e25112a7a5a6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:23.613389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25af28-ff97-4c6b-b40b-4f8218d3918e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:23.614098 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25af28-ff97-4c6b-b40b-4f8218d3918e admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:23.614406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25af28-ff97-4c6b-b40b-4f8218d3918e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:23.616570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25af28-ff97-4c6b-b40b-4f8218d3918e admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:23.617590 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25af28-ff97-4c6b-b40b-4f8218d3918e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:23.620481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25af28-ff97-4c6b-b40b-4f8218d3918e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:23.643838 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1073/2155] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:23 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:19:23.658161 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8560bcce-66c7-4d48-b008-14ad18a97d2f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:23.661435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8560bcce-66c7-4d48-b008-14ad18a97d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:23.739926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1083/2156] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:19:23 2022] POST /identity/v3/users => generated 294 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:19:23.741216 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25af28-ff97-4c6b-b40b-4f8218d3918e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:23.758233 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25af28-ff97-4c6b-b40b-4f8218d3918e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:23.765956 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1e346-e126-4751-ae6a-cbf3dd3f513d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:23.766967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1e346-e126-4751-ae6a-cbf3dd3f513d admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:23.776670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1e346-e126-4751-ae6a-cbf3dd3f513d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:23.776670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1e346-e126-4751-ae6a-cbf3dd3f513d admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:23.776670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c1e346-e126-4751-ae6a-cbf3dd3f513d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:23.776670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c1e346-e126-4751-ae6a-cbf3dd3f513d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:23.794551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1084/2157] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:19:23.798804 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1e346-e126-4751-ae6a-cbf3dd3f513d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:23.799075 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8560bcce-66c7-4d48-b008-14ad18a97d2f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:23.808301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d1a8f-061e-429b-bedc-00e4db6d01bb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61459e15d327405b88a2fdfc1fc0fecd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:23.810772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d1a8f-061e-429b-bedc-00e4db6d01bb service glance] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:23.810772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d1a8f-061e-429b-bedc-00e4db6d01bb service glance] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:23.810772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d1a8f-061e-429b-bedc-00e4db6d01bb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:23.811303 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1e346-e126-4751-ae6a-cbf3dd3f513d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:23.823016 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d292a8-83b9-401f-bb60-bb9f947bf317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:23.823797 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d292a8-83b9-401f-bb60-bb9f947bf317 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:23.824188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d292a8-83b9-401f-bb60-bb9f947bf317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:23.824695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d292a8-83b9-401f-bb60-bb9f947bf317 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:23.825629 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d292a8-83b9-401f-bb60-bb9f947bf317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:23.827919 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d292a8-83b9-401f-bb60-bb9f947bf317 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:23.840075 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d1a8f-061e-429b-bedc-00e4db6d01bb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:23.842595 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747d1a8f-061e-429b-bedc-00e4db6d01bb service glance] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:23.843210 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1085/2158] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:19:23.850412 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d292a8-83b9-401f-bb60-bb9f947bf317 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:23.860726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1074/2159] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:19:23 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:19:23.866505 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d292a8-83b9-401f-bb60-bb9f947bf317 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:23.873011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121e211-ac29-4021-a61d-fe0a82e2b495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:23.874150 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121e211-ac29-4021-a61d-fe0a82e2b495 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:23.877194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121e211-ac29-4021-a61d-fe0a82e2b495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:23.877194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121e211-ac29-4021-a61d-fe0a82e2b495 admin admin] PATH_INFO: `/v3/projects/8f837f55aa7b41d9b2722684c1e2796d/users/f0be96d1f1d2405499ab8185a27eac44/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:23.910833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121e211-ac29-4021-a61d-fe0a82e2b495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f837f55aa7b41d9b2722684c1e2796d, user_id=f0be96d1f1d2405499ab8185a27eac44, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:23.913822 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121e211-ac29-4021-a61d-fe0a82e2b495 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:24.037326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1086/2160] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:23 2022] PUT /identity/v3/projects/8f837f55aa7b41d9b2722684c1e2796d/users/f0be96d1f1d2405499ab8185a27eac44/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:24.047192 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c6106-35db-429f-9eb0-83aa75a74120 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:24.048427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c6106-35db-429f-9eb0-83aa75a74120 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:24.048806 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c6106-35db-429f-9eb0-83aa75a74120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:24.133255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-856c6106-35db-429f-9eb0-83aa75a74120 None None] MFA Rules not processed for user `f0be96d1f1d2405499ab8185a27eac44`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:24.136830 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-856c6106-35db-429f-9eb0-83aa75a74120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:24.336611 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:24.361108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1075/2161] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:24.366074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121e211-ac29-4021-a61d-fe0a82e2b495 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:24.553044 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121e211-ac29-4021-a61d-fe0a82e2b495 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:24.560271 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6663ea-1755-4200-941c-2ec93303b749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:24.561112 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6663ea-1755-4200-941c-2ec93303b749 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:24.561522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6663ea-1755-4200-941c-2ec93303b749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:24.561893 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6663ea-1755-4200-941c-2ec93303b749 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:24.563015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6663ea-1755-4200-941c-2ec93303b749 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:24.565306 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6663ea-1755-4200-941c-2ec93303b749 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:24.732447 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1087/2162] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:19:24 2022] POST /identity/v3/projects => generated 387 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:24.736996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c6106-35db-429f-9eb0-83aa75a74120 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:24.754001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c6106-35db-429f-9eb0-83aa75a74120 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:24.834458 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745b128-8db3-42e1-9695-9b334d1c1ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:24.836056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745b128-8db3-42e1-9695-9b334d1c1ca3 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:24.837384 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745b128-8db3-42e1-9695-9b334d1c1ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:24.837384 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a745b128-8db3-42e1-9695-9b334d1c1ca3 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:24.837593 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a745b128-8db3-42e1-9695-9b334d1c1ca3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:24.839577 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a745b128-8db3-42e1-9695-9b334d1c1ca3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:24.954796 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1076/2163] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:19:24 2022] POST /identity/v3/users => generated 300 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:24.956012 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6663ea-1755-4200-941c-2ec93303b749 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:24.972619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6663ea-1755-4200-941c-2ec93303b749 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:24.988028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61228f-5aae-45b1-b88f-4f53e0a1eb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:24.988715 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61228f-5aae-45b1-b88f-4f53e0a1eb96 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:24.989067 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61228f-5aae-45b1-b88f-4f53e0a1eb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:24.989425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61228f-5aae-45b1-b88f-4f53e0a1eb96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:24.991172 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b61228f-5aae-45b1-b88f-4f53e0a1eb96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:24.994388 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b61228f-5aae-45b1-b88f-4f53e0a1eb96 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:25.017642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1088/2164] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:25.021708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745b128-8db3-42e1-9695-9b334d1c1ca3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:25.048158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a745b128-8db3-42e1-9695-9b334d1c1ca3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:25.055920 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c7422-873a-4a6b-aca6-b5bb843216a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:25.057085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c7422-873a-4a6b-aca6-b5bb843216a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:25.057457 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c7422-873a-4a6b-aca6-b5bb843216a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:25.057808 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c7422-873a-4a6b-aca6-b5bb843216a5 admin admin] PATH_INFO: `/v3/projects/3d917fabe9c04481a0affbcda453266f/users/a7fcd3f0c5954dfdb9dc808b3cc7717b/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:25.083344 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c7422-873a-4a6b-aca6-b5bb843216a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d917fabe9c04481a0affbcda453266f, user_id=a7fcd3f0c5954dfdb9dc808b3cc7717b, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:25.086513 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176c7422-873a-4a6b-aca6-b5bb843216a5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:25.165457 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:25.185440 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1077/2165] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:25 2022] PUT /identity/v3/projects/3d917fabe9c04481a0affbcda453266f/users/a7fcd3f0c5954dfdb9dc808b3cc7717b/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:25.190139 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61228f-5aae-45b1-b88f-4f53e0a1eb96 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:25.202400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61228f-5aae-45b1-b88f-4f53e0a1eb96 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:25.317137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901f0c5-3509-463e-897a-50b9cc0d75b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:25.317137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901f0c5-3509-463e-897a-50b9cc0d75b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:25.317137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901f0c5-3509-463e-897a-50b9cc0d75b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:25.317137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901f0c5-3509-463e-897a-50b9cc0d75b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:25.317462 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901f0c5-3509-463e-897a-50b9cc0d75b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:25.319257 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901f0c5-3509-463e-897a-50b9cc0d75b5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:25.334510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1089/2166] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:25.338507 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c7422-873a-4a6b-aca6-b5bb843216a5 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:25.351678 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176c7422-873a-4a6b-aca6-b5bb843216a5 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:25.358812 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715618a2-fb3d-43c2-adaa-1d0b13b52e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:25.360137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715618a2-fb3d-43c2-adaa-1d0b13b52e84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:25.360627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715618a2-fb3d-43c2-adaa-1d0b13b52e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:25.360969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715618a2-fb3d-43c2-adaa-1d0b13b52e84 admin admin] PATH_INFO: `/v3/projects/3d917fabe9c04481a0affbcda453266f/users/a7fcd3f0c5954dfdb9dc808b3cc7717b/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:25.363374 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715618a2-fb3d-43c2-adaa-1d0b13b52e84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d917fabe9c04481a0affbcda453266f, user_id=a7fcd3f0c5954dfdb9dc808b3cc7717b, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:25.368271 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715618a2-fb3d-43c2-adaa-1d0b13b52e84 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:25.381590 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1078/2167] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:25 2022] PUT /identity/v3/projects/3d917fabe9c04481a0affbcda453266f/users/a7fcd3f0c5954dfdb9dc808b3cc7717b/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:25.394696 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be73ed-a093-45b8-99be-59bb4fd624ec None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:25.395052 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be73ed-a093-45b8-99be-59bb4fd624ec None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:25.395339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be73ed-a093-45b8-99be-59bb4fd624ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:25.434269 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-72be73ed-a093-45b8-99be-59bb4fd624ec None None] MFA Rules not processed for user `a7fcd3f0c5954dfdb9dc808b3cc7717b`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:25.436308 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-72be73ed-a093-45b8-99be-59bb4fd624ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:25.536388 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1090/2168] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:25.548693 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715618a2-fb3d-43c2-adaa-1d0b13b52e84 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:25.550518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-715618a2-fb3d-43c2-adaa-1d0b13b52e84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:25.629982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715618a2-fb3d-43c2-adaa-1d0b13b52e84 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:25.654018 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f0df2-ee64-40dd-9f2d-1cae10d7cd95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:25.655012 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f0df2-ee64-40dd-9f2d-1cae10d7cd95 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:25.655294 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f0df2-ee64-40dd-9f2d-1cae10d7cd95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:25.655576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f0df2-ee64-40dd-9f2d-1cae10d7cd95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:25.674743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f0df2-ee64-40dd-9f2d-1cae10d7cd95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:25.676515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f0df2-ee64-40dd-9f2d-1cae10d7cd95 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:25.712464 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1079/2169] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:19:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:19:26.299842 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:27.356144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be73ed-a093-45b8-99be-59bb4fd624ec None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:27.505250 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be73ed-a093-45b8-99be-59bb4fd624ec None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:27.516003 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0582bc-024a-4106-93a8-02208a7feceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:27.517330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0582bc-024a-4106-93a8-02208a7feceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:27.517850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0582bc-024a-4106-93a8-02208a7feceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:27.518358 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0582bc-024a-4106-93a8-02208a7feceb admin admin] PATH_INFO: `/v3/users/f0be96d1f1d2405499ab8185a27eac44` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:27.524052 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0582bc-024a-4106-93a8-02208a7feceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0be96d1f1d2405499ab8185a27eac44)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:27.531789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0582bc-024a-4106-93a8-02208a7feceb admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:27.685308 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:27.787878 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1a0582bc-024a-4106-93a8-02208a7feceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0be96d1f1d2405499ab8185a27eac44'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:27.811137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1a0582bc-024a-4106-93a8-02208a7feceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0be96d1f1d2405499ab8185a27eac44'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:27.812150 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1a0582bc-024a-4106-93a8-02208a7feceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0be96d1f1d2405499ab8185a27eac44'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:27.812958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1a0582bc-024a-4106-93a8-02208a7feceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0be96d1f1d2405499ab8185a27eac44'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:27.820635 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1a0582bc-024a-4106-93a8-02208a7feceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0be96d1f1d2405499ab8185a27eac44'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:27.854068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:27.930311 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1091/2170] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:19:27 2022] DELETE /identity/v3/users/f0be96d1f1d2405499ab8185a27eac44 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:27.935643 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89334685-cc40-4a1c-a1de-f05fe9741d91 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:27.936158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89334685-cc40-4a1c-a1de-f05fe9741d91 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:27.936586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89334685-cc40-4a1c-a1de-f05fe9741d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:27.994698 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-89334685-cc40-4a1c-a1de-f05fe9741d91 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:27.999051 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-89334685-cc40-4a1c-a1de-f05fe9741d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:28.140033 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1080/2171] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:28.154230 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0582bc-024a-4106-93a8-02208a7feceb admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:28.156785 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1a0582bc-024a-4106-93a8-02208a7feceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:28.282639 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0582bc-024a-4106-93a8-02208a7feceb admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:28.297636 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1b847-832d-4857-823b-f095695cb38e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:28.298575 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1b847-832d-4857-823b-f095695cb38e service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:28.299018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1b847-832d-4857-823b-f095695cb38e service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:28.299559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1b847-832d-4857-823b-f095695cb38e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:28.311701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1b847-832d-4857-823b-f095695cb38e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:28.313324 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1b847-832d-4857-823b-f095695cb38e service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:28.327545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1092/2172] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:19:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:19:29.358580 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19db67-b3c0-4730-8459-edfa6d6fbe1d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:29.358580 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19db67-b3c0-4730-8459-edfa6d6fbe1d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:29.358580 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19db67-b3c0-4730-8459-edfa6d6fbe1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:29.378271 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1b847-832d-4857-823b-f095695cb38e service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:29.379973 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1cd1b847-832d-4857-823b-f095695cb38e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:29.424597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1b847-832d-4857-823b-f095695cb38e service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:29.441865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2d19db67-b3c0-4730-8459-edfa6d6fbe1d None None] MFA Rules not processed for user `114d227e9645409e9082cf9b09b05c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:29.443156 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2d19db67-b3c0-4730-8459-edfa6d6fbe1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:29.448890 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a05342-afcb-47f7-a02d-193aff660dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:29.451099 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a05342-afcb-47f7-a02d-193aff660dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:29.451099 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a05342-afcb-47f7-a02d-193aff660dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:29.451099 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a05342-afcb-47f7-a02d-193aff660dcd admin admin] PATH_INFO: `/v3/projects/8f837f55aa7b41d9b2722684c1e2796d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:29.453166 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a05342-afcb-47f7-a02d-193aff660dcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f837f55aa7b41d9b2722684c1e2796d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:29.459115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a05342-afcb-47f7-a02d-193aff660dcd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:29.552648 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1081/2173] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:29.554116 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ddf83-aaf5-47c0-8684-7a257cc10ec6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:29.554512 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ddf83-aaf5-47c0-8684-7a257cc10ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:29.554863 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2ddf83-aaf5-47c0-8684-7a257cc10ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:29.602423 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0b2ddf83-aaf5-47c0-8684-7a257cc10ec6 None None] MFA Rules not processed for user `3ec36b2f17324fe699593c3aebb6c39c`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:29.605258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0b2ddf83-aaf5-47c0-8684-7a257cc10ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:29.620911 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-14a05342-afcb-47f7-a02d-193aff660dcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f837f55aa7b41d9b2722684c1e2796d', 'request_id': 'req-14a05342-afcb-47f7-a02d-193aff660dcd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:29.628928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-14a05342-afcb-47f7-a02d-193aff660dcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f837f55aa7b41d9b2722684c1e2796d', 'request_id': 'req-14a05342-afcb-47f7-a02d-193aff660dcd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:29.636024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-14a05342-afcb-47f7-a02d-193aff660dcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f837f55aa7b41d9b2722684c1e2796d', 'request_id': 'req-14a05342-afcb-47f7-a02d-193aff660dcd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:29.703054 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1082/2174] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:29.754704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-14a05342-afcb-47f7-a02d-193aff660dcd admin admin] The token cache is being invalidate because project 8f837f55aa7b41d9b2722684c1e2796d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:19:29.755811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-14a05342-afcb-47f7-a02d-193aff660dcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:29.756088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-14a05342-afcb-47f7-a02d-193aff660dcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:29.757996 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1093/2175] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:19:29 2022] DELETE /identity/v3/projects/8f837f55aa7b41d9b2722684c1e2796d => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:29.763915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ddf83-aaf5-47c0-8684-7a257cc10ec6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:29.776065 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:29.838646 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:29.844527 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2ddf83-aaf5-47c0-8684-7a257cc10ec6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:29.849292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9aeaf-ce09-4b31-a8da-fb9ac5c7a04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:29.850184 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9aeaf-ce09-4b31-a8da-fb9ac5c7a04c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:29.850481 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9aeaf-ce09-4b31-a8da-fb9ac5c7a04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:29.850783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9aeaf-ce09-4b31-a8da-fb9ac5c7a04c admin admin] PATH_INFO: `/v3/users/a7fcd3f0c5954dfdb9dc808b3cc7717b` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:29.852000 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9aeaf-ce09-4b31-a8da-fb9ac5c7a04c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7fcd3f0c5954dfdb9dc808b3cc7717b)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:29.853930 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9aeaf-ce09-4b31-a8da-fb9ac5c7a04c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:30.096081 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ddc9aeaf-ce09-4b31-a8da-fb9ac5c7a04c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7fcd3f0c5954dfdb9dc808b3cc7717b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:30.109179 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ddc9aeaf-ce09-4b31-a8da-fb9ac5c7a04c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7fcd3f0c5954dfdb9dc808b3cc7717b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:30.195959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ddc9aeaf-ce09-4b31-a8da-fb9ac5c7a04c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7fcd3f0c5954dfdb9dc808b3cc7717b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:30.196722 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ddc9aeaf-ce09-4b31-a8da-fb9ac5c7a04c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7fcd3f0c5954dfdb9dc808b3cc7717b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:30.197416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ddc9aeaf-ce09-4b31-a8da-fb9ac5c7a04c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7fcd3f0c5954dfdb9dc808b3cc7717b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:30.212751 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1083/2176] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:19:29 2022] DELETE /identity/v3/users/a7fcd3f0c5954dfdb9dc808b3cc7717b => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:19:30.756428 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a05342-afcb-47f7-a02d-193aff660dcd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:30.758561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-14a05342-afcb-47f7-a02d-193aff660dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:30.772786 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a05342-afcb-47f7-a02d-193aff660dcd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:30.851033 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a7c9b-b12f-4ced-9f58-53fc2b3c7388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:30.851907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a7c9b-b12f-4ced-9f58-53fc2b3c7388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:30.852195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a7c9b-b12f-4ced-9f58-53fc2b3c7388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:30.853600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a7c9b-b12f-4ced-9f58-53fc2b3c7388 admin admin] PATH_INFO: `/v3/projects/3d917fabe9c04481a0affbcda453266f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:30.853712 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a7c9b-b12f-4ced-9f58-53fc2b3c7388 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d917fabe9c04481a0affbcda453266f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:30.855806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a7c9b-b12f-4ced-9f58-53fc2b3c7388 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:30.993527 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-492a7c9b-b12f-4ced-9f58-53fc2b3c7388 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d917fabe9c04481a0affbcda453266f', 'request_id': 'req-492a7c9b-b12f-4ced-9f58-53fc2b3c7388'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:31.001533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-492a7c9b-b12f-4ced-9f58-53fc2b3c7388 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d917fabe9c04481a0affbcda453266f', 'request_id': 'req-492a7c9b-b12f-4ced-9f58-53fc2b3c7388'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:31.008007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-492a7c9b-b12f-4ced-9f58-53fc2b3c7388 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d917fabe9c04481a0affbcda453266f', 'request_id': 'req-492a7c9b-b12f-4ced-9f58-53fc2b3c7388'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:31.098115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-492a7c9b-b12f-4ced-9f58-53fc2b3c7388 admin admin] The token cache is being invalidate because project 3d917fabe9c04481a0affbcda453266f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:19:31.098427 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-492a7c9b-b12f-4ced-9f58-53fc2b3c7388 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:31.099051 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-492a7c9b-b12f-4ced-9f58-53fc2b3c7388 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:19:31.100580 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1094/2177] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:19:30 2022] DELETE /identity/v3/projects/3d917fabe9c04481a0affbcda453266f => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:19:31.105571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28381520-1679-4765-9e54-2d32a22418d2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:31.106073 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28381520-1679-4765-9e54-2d32a22418d2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:31.106196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28381520-1679-4765-9e54-2d32a22418d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:31.128481 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-28381520-1679-4765-9e54-2d32a22418d2 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:31.129691 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-28381520-1679-4765-9e54-2d32a22418d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:31.192912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1084/2178] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:31.203121 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7851ac-948a-43f2-94dd-86e80ab3db38 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:31.203420 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7851ac-948a-43f2-94dd-86e80ab3db38 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:31.203763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7851ac-948a-43f2-94dd-86e80ab3db38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:31.229597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-aa7851ac-948a-43f2-94dd-86e80ab3db38 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:31.230843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-aa7851ac-948a-43f2-94dd-86e80ab3db38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:31.246948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1095/2179] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:31.252053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28381520-1679-4765-9e54-2d32a22418d2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:31.266860 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28381520-1679-4765-9e54-2d32a22418d2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:31.271885 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c2118-de8e-48cb-9376-c657ba4d4563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:31.272973 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c2118-de8e-48cb-9376-c657ba4d4563 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:31.273264 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c2118-de8e-48cb-9376-c657ba4d4563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:31.273541 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c2118-de8e-48cb-9376-c657ba4d4563 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:31.274313 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c2118-de8e-48cb-9376-c657ba4d4563 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:19:31.274651 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c2118-de8e-48cb-9376-c657ba4d4563 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:31.276275 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c2118-de8e-48cb-9376-c657ba4d4563 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:31.288010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:31.289645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1085/2180] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:19:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:19:31.293676 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7851ac-948a-43f2-94dd-86e80ab3db38 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:31.303805 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7851ac-948a-43f2-94dd-86e80ab3db38 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:31.309297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d09c5c-3ec1-4975-9331-75884e82dbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:31.310849 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d09c5c-3ec1-4975-9331-75884e82dbf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:31.311142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d09c5c-3ec1-4975-9331-75884e82dbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:31.311428 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d09c5c-3ec1-4975-9331-75884e82dbf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:31.312149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d09c5c-3ec1-4975-9331-75884e82dbf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:31.314647 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d09c5c-3ec1-4975-9331-75884e82dbf7 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:31.413773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:31.421040 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1096/2181] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:19:31 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:31.425143 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c2118-de8e-48cb-9376-c657ba4d4563 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:31.435142 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c2118-de8e-48cb-9376-c657ba4d4563 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:31.494559 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8f696-780f-4d2c-8317-d1c57ffb835e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:31.495230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8f696-780f-4d2c-8317-d1c57ffb835e admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:31.495512 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8f696-780f-4d2c-8317-d1c57ffb835e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:31.495788 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8f696-780f-4d2c-8317-d1c57ffb835e admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:31.496482 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8f696-780f-4d2c-8317-d1c57ffb835e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:31.498530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8f696-780f-4d2c-8317-d1c57ffb835e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:31.625810 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1086/2182] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:19:31 2022] POST /identity/v3/users => generated 287 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:19:31.629758 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d09c5c-3ec1-4975-9331-75884e82dbf7 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:31.644317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d09c5c-3ec1-4975-9331-75884e82dbf7 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:31.668819 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292bc24-a673-497b-b5ca-ccb9df05c04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:31.670117 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292bc24-a673-497b-b5ca-ccb9df05c04e admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:31.670629 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292bc24-a673-497b-b5ca-ccb9df05c04e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:31.671040 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9292bc24-a673-497b-b5ca-ccb9df05c04e admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:31.671822 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292bc24-a673-497b-b5ca-ccb9df05c04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:31.673522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9292bc24-a673-497b-b5ca-ccb9df05c04e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:31.686360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1097/2183] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:19:31.690790 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8f696-780f-4d2c-8317-d1c57ffb835e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:31.704472 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8f696-780f-4d2c-8317-d1c57ffb835e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:31.710571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a757901-6082-48ed-adbb-c84b9c20db9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:31.711239 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a757901-6082-48ed-adbb-c84b9c20db9a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:31.711526 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a757901-6082-48ed-adbb-c84b9c20db9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:31.711804 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a757901-6082-48ed-adbb-c84b9c20db9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:31.712533 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a757901-6082-48ed-adbb-c84b9c20db9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:31.714404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a757901-6082-48ed-adbb-c84b9c20db9a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:31.729531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1087/2184] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:19:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:19:31.731314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292bc24-a673-497b-b5ca-ccb9df05c04e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:31.748822 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9292bc24-a673-497b-b5ca-ccb9df05c04e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:31.755826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82beb1b-1418-47d7-bd3c-5699db2ead82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:31.759703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82beb1b-1418-47d7-bd3c-5699db2ead82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:31.760502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82beb1b-1418-47d7-bd3c-5699db2ead82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:31.760502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82beb1b-1418-47d7-bd3c-5699db2ead82 admin admin] PATH_INFO: `/v3/projects/46222aa2184c4a6d82b65d3c703c8bd7/users/8c6ef9f5f6ac4b6289aa1aa9e16ce4bb/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:31.824148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82beb1b-1418-47d7-bd3c-5699db2ead82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46222aa2184c4a6d82b65d3c703c8bd7, user_id=8c6ef9f5f6ac4b6289aa1aa9e16ce4bb, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:31.828671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82beb1b-1418-47d7-bd3c-5699db2ead82 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:31.916443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1098/2185] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:19:31 2022] PUT /identity/v3/projects/46222aa2184c4a6d82b65d3c703c8bd7/users/8c6ef9f5f6ac4b6289aa1aa9e16ce4bb/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:19:31.924629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0ef74-9be5-42ef-8182-44a7c352a329 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:31.924999 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0ef74-9be5-42ef-8182-44a7c352a329 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:31.925351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0ef74-9be5-42ef-8182-44a7c352a329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:31.962349 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-afe0ef74-9be5-42ef-8182-44a7c352a329 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:31.973894 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-afe0ef74-9be5-42ef-8182-44a7c352a329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:32.086998 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1088/2186] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:32.097885 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82beb1b-1418-47d7-bd3c-5699db2ead82 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:32.100520 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d82beb1b-1418-47d7-bd3c-5699db2ead82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:32.112339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0ef74-9be5-42ef-8182-44a7c352a329 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:32.114034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-afe0ef74-9be5-42ef-8182-44a7c352a329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:32.220438 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe0ef74-9be5-42ef-8182-44a7c352a329 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:32.224377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82beb1b-1418-47d7-bd3c-5699db2ead82 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:32.224788 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb00810-8ddd-4b2e-960c-ef03f756c4e4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61459e15d327405b88a2fdfc1fc0fecd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:32.225748 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb00810-8ddd-4b2e-960c-ef03f756c4e4 service glance] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:32.225748 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb00810-8ddd-4b2e-960c-ef03f756c4e4 service glance] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:32.226021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb00810-8ddd-4b2e-960c-ef03f756c4e4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:32.228339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3eb00810-8ddd-4b2e-960c-ef03f756c4e4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:32.230344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4e95f-3a3f-42b6-8b4f-df63eba6654d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:32.231290 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4e95f-3a3f-42b6-8b4f-df63eba6654d service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:32.232814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4e95f-3a3f-42b6-8b4f-df63eba6654d service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:32.233187 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4e95f-3a3f-42b6-8b4f-df63eba6654d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:32.243847 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4e95f-3a3f-42b6-8b4f-df63eba6654d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:32.245418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4e95f-3a3f-42b6-8b4f-df63eba6654d service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:32.257701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1099/2187] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:19:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:19:32.330157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb00810-8ddd-4b2e-960c-ef03f756c4e4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:32.331848 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb00810-8ddd-4b2e-960c-ef03f756c4e4 service glance] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:32.342681 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1089/2188] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:19:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:19:32.599086 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:32.694505 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ff8ba-5398-4617-9da9-c57116274b3d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:32.694914 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ff8ba-5398-4617-9da9-c57116274b3d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:32.695260 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ff8ba-5398-4617-9da9-c57116274b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:32.728497 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-446ff8ba-5398-4617-9da9-c57116274b3d None None] MFA Rules not processed for user `98f58f91d17949198e66883d234a9e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:32.730724 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-446ff8ba-5398-4617-9da9-c57116274b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:32.882463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb00810-8ddd-4b2e-960c-ef03f756c4e4 service glance] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:32.884227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3eb00810-8ddd-4b2e-960c-ef03f756c4e4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:32.886839 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1100/2189] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jun 24 04:19:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:33.020788 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb00810-8ddd-4b2e-960c-ef03f756c4e4 service glance] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:33.033483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d67909-579f-4f67-ba46-f75faf2213cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8e1f1e3c7e42cc9fa6f82aaac55129', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:33.033962 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d67909-579f-4f67-ba46-f75faf2213cc service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:33.036113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d67909-579f-4f67-ba46-f75faf2213cc service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:33.037322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d67909-579f-4f67-ba46-f75faf2213cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:33.065420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d67909-579f-4f67-ba46-f75faf2213cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:33.069576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d67909-579f-4f67-ba46-f75faf2213cc service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:33.104609 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1090/2190] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jun 24 04:19:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 04:19:33.227913 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=2,keystone:SELECT=92,keystone:INSERT=6 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:33.774652 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:34.045646 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:34.273402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:34.931321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:35.009898 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:35.625963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55433d74-0da0-43ee-80d6-17c32809d2a9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:35.626619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55433d74-0da0-43ee-80d6-17c32809d2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:35.626619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55433d74-0da0-43ee-80d6-17c32809d2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:35.699372 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-55433d74-0da0-43ee-80d6-17c32809d2a9 None None] MFA Rules not processed for user `8c6ef9f5f6ac4b6289aa1aa9e16ce4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:35.702891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-55433d74-0da0-43ee-80d6-17c32809d2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:35.883483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1101/2191] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:35.896991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d67909-579f-4f67-ba46-f75faf2213cc service swift] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:35.898912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-19d67909-579f-4f67-ba46-f75faf2213cc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:36.072873 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d67909-579f-4f67-ba46-f75faf2213cc service swift] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:36.079791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12bea0a-04ad-42d1-aba8-6af23a6e67d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:36.080439 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bea0a-04ad-42d1-aba8-6af23a6e67d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:36.080728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bea0a-04ad-42d1-aba8-6af23a6e67d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:36.081002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12bea0a-04ad-42d1-aba8-6af23a6e67d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:36.093112 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12bea0a-04ad-42d1-aba8-6af23a6e67d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:36.099345 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12bea0a-04ad-42d1-aba8-6af23a6e67d1 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:36.157860 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1091/2192] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:19:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:19:37.006352 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:38.226278 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33423acb-5f08-4647-9283-ce23bdab5dfe None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:38.227530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33423acb-5f08-4647-9283-ce23bdab5dfe None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:38.228135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33423acb-5f08-4647-9283-ce23bdab5dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:38.251681 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-33423acb-5f08-4647-9283-ce23bdab5dfe None None] MFA Rules not processed for user `8c6ef9f5f6ac4b6289aa1aa9e16ce4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:38.255912 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-33423acb-5f08-4647-9283-ce23bdab5dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:38.261693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1102/2193] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:43.577939 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78372633-40d9-465a-9e16-d16f2a13974d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:43.580109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78372633-40d9-465a-9e16-d16f2a13974d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:43.580557 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78372633-40d9-465a-9e16-d16f2a13974d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:43.625514 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-78372633-40d9-465a-9e16-d16f2a13974d None None] MFA Rules not processed for user `98f58f91d17949198e66883d234a9e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:43.630283 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-78372633-40d9-465a-9e16-d16f2a13974d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:43.655698 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1092/2194] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jun 24 04:19:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:43.668127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33423acb-5f08-4647-9283-ce23bdab5dfe None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:43.696511 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33423acb-5f08-4647-9283-ce23bdab5dfe None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:43.703351 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8c4f0-19cc-4c55-a688-3ec0fd65a3b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8e1f1e3c7e42cc9fa6f82aaac55129', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:43.704236 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8c4f0-19cc-4c55-a688-3ec0fd65a3b1 service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:43.704684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8c4f0-19cc-4c55-a688-3ec0fd65a3b1 service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:43.705200 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8c4f0-19cc-4c55-a688-3ec0fd65a3b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:43.713791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:43.724681 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8c4f0-19cc-4c55-a688-3ec0fd65a3b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:43.726678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8c4f0-19cc-4c55-a688-3ec0fd65a3b1 service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:43.755187 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1103/2195] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jun 24 04:19:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 04:19:49.203523 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d3f63-b7ac-4d43-842b-abc4dab23cf3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:49.205095 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d3f63-b7ac-4d43-842b-abc4dab23cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:49.205766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d3f63-b7ac-4d43-842b-abc4dab23cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:49.237851 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7d6d3f63-b7ac-4d43-842b-abc4dab23cf3 None None] MFA Rules not processed for user `4fb36d1e057543a3abcf6d6c2564d9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:49.244012 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7d6d3f63-b7ac-4d43-842b-abc4dab23cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:49.265963 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1093/2196] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:19:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:53.760266 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:19:58.383466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b35fba-8eca-4090-9305-39c6f9cda590 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:58.384543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b35fba-8eca-4090-9305-39c6f9cda590 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:58.385010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b35fba-8eca-4090-9305-39c6f9cda590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:58.437462 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-16b35fba-8eca-4090-9305-39c6f9cda590 None None] MFA Rules not processed for user `98f58f91d17949198e66883d234a9e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:19:58.439077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-16b35fba-8eca-4090-9305-39c6f9cda590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:19:58.458397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1104/2197] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jun 24 04:19:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:19:58.517191 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d3f63-b7ac-4d43-842b-abc4dab23cf3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:19:58.538089 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d3f63-b7ac-4d43-842b-abc4dab23cf3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:19:58.553893 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35063d-0ad4-4726-a162-70b46857c488 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8e1f1e3c7e42cc9fa6f82aaac55129', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:19:58.554654 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35063d-0ad4-4726-a162-70b46857c488 service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:19:58.555216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35063d-0ad4-4726-a162-70b46857c488 service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:19:58.555703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b35063d-0ad4-4726-a162-70b46857c488 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:19:58.578583 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35063d-0ad4-4726-a162-70b46857c488 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:19:58.580627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b35063d-0ad4-4726-a162-70b46857c488 service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:19:58.603299 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1094/2198] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jun 24 04:19:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 04:20:00.743939 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cab776-decd-420e-8662-efa1d8f53067 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:00.744532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cab776-decd-420e-8662-efa1d8f53067 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:00.744532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cab776-decd-420e-8662-efa1d8f53067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:00.778015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c9cab776-decd-420e-8662-efa1d8f53067 None None] MFA Rules not processed for user `b158078d039a4979a9c0cf2be26c4b14`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:20:00.782305 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c9cab776-decd-420e-8662-efa1d8f53067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:00.884908 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1105/2199] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:20:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:20:08.587053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=67 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:20:10.165115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35063d-0ad4-4726-a162-70b46857c488 service swift] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:10.166720 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1b35063d-0ad4-4726-a162-70b46857c488 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:10.223834 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b35063d-0ad4-4726-a162-70b46857c488 service swift] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:10.228798 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51beaeb7-4e2c-48e7-b5af-2c4e07d94a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:10.229827 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51beaeb7-4e2c-48e7-b5af-2c4e07d94a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:10.231491 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51beaeb7-4e2c-48e7-b5af-2c4e07d94a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:10.231897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51beaeb7-4e2c-48e7-b5af-2c4e07d94a98 admin admin] PATH_INFO: `/v3/users/b45a74c5a7d145bc8371ada44fbf9916` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:10.233224 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51beaeb7-4e2c-48e7-b5af-2c4e07d94a98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b45a74c5a7d145bc8371ada44fbf9916)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:10.235272 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51beaeb7-4e2c-48e7-b5af-2c4e07d94a98 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:10.433623 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-51beaeb7-4e2c-48e7-b5af-2c4e07d94a98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b45a74c5a7d145bc8371ada44fbf9916'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:10.451090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-51beaeb7-4e2c-48e7-b5af-2c4e07d94a98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b45a74c5a7d145bc8371ada44fbf9916'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:10.516008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-51beaeb7-4e2c-48e7-b5af-2c4e07d94a98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b45a74c5a7d145bc8371ada44fbf9916'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:10.516671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-51beaeb7-4e2c-48e7-b5af-2c4e07d94a98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b45a74c5a7d145bc8371ada44fbf9916'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:10.517351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-51beaeb7-4e2c-48e7-b5af-2c4e07d94a98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b45a74c5a7d145bc8371ada44fbf9916'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:10.537663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1095/2200] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:20:10 2022] DELETE /identity/v3/users/b45a74c5a7d145bc8371ada44fbf9916 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:20:10.542056 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82463e51-24a6-4b23-b6ea-c19e20080552 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:10.543451 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82463e51-24a6-4b23-b6ea-c19e20080552 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:10.543451 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82463e51-24a6-4b23-b6ea-c19e20080552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:10.575013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-82463e51-24a6-4b23-b6ea-c19e20080552 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:20:10.576070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-82463e51-24a6-4b23-b6ea-c19e20080552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:10.662831 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1106/2201] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:20:10.672835 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51beaeb7-4e2c-48e7-b5af-2c4e07d94a98 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:10.674517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-51beaeb7-4e2c-48e7-b5af-2c4e07d94a98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:10.795135 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51beaeb7-4e2c-48e7-b5af-2c4e07d94a98 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:10.799887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97146e07-651a-472e-b696-3f3bac26c069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:10.800958 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97146e07-651a-472e-b696-3f3bac26c069 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:10.801323 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97146e07-651a-472e-b696-3f3bac26c069 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:10.801677 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97146e07-651a-472e-b696-3f3bac26c069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:10.814562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97146e07-651a-472e-b696-3f3bac26c069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:10.814562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97146e07-651a-472e-b696-3f3bac26c069 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:10.826739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1096/2202] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:20:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:20:11.453492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82463e51-24a6-4b23-b6ea-c19e20080552 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:11.453492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-82463e51-24a6-4b23-b6ea-c19e20080552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:11.464185 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82463e51-24a6-4b23-b6ea-c19e20080552 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:11.479659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490ee13-d56e-4305-aca9-53d1439b2c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:11.480879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490ee13-d56e-4305-aca9-53d1439b2c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:11.481171 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490ee13-d56e-4305-aca9-53d1439b2c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:11.481612 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490ee13-d56e-4305-aca9-53d1439b2c1b admin admin] PATH_INFO: `/v3/projects/48a2b0b627024d2a9d3309b4e5e4bbf5` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:11.483053 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490ee13-d56e-4305-aca9-53d1439b2c1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=48a2b0b627024d2a9d3309b4e5e4bbf5)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:11.485467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490ee13-d56e-4305-aca9-53d1439b2c1b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:11.618498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9490ee13-d56e-4305-aca9-53d1439b2c1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48a2b0b627024d2a9d3309b4e5e4bbf5', 'request_id': 'req-9490ee13-d56e-4305-aca9-53d1439b2c1b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:11.626133 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9490ee13-d56e-4305-aca9-53d1439b2c1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48a2b0b627024d2a9d3309b4e5e4bbf5', 'request_id': 'req-9490ee13-d56e-4305-aca9-53d1439b2c1b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:11.632663 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9490ee13-d56e-4305-aca9-53d1439b2c1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48a2b0b627024d2a9d3309b4e5e4bbf5', 'request_id': 'req-9490ee13-d56e-4305-aca9-53d1439b2c1b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:11.696960 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:20:11.701324 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9490ee13-d56e-4305-aca9-53d1439b2c1b admin admin] The token cache is being invalidate because project 48a2b0b627024d2a9d3309b4e5e4bbf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:20:11.701616 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9490ee13-d56e-4305-aca9-53d1439b2c1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:11.702255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9490ee13-d56e-4305-aca9-53d1439b2c1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:11.703837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1107/2203] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:20:11 2022] DELETE /identity/v3/projects/48a2b0b627024d2a9d3309b4e5e4bbf5 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:20:11.709028 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531f4f3-711a-4b0c-a23c-15db91364da9 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:11.709555 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531f4f3-711a-4b0c-a23c-15db91364da9 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:11.709657 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531f4f3-711a-4b0c-a23c-15db91364da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:11.748587 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:20:11.760450 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c531f4f3-711a-4b0c-a23c-15db91364da9 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:20:11.761807 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c531f4f3-711a-4b0c-a23c-15db91364da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:11.824785 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1097/2204] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:20:11.833127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24d2b6-d7a5-4c18-8aab-0c7eec5279f2 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:11.833414 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24d2b6-d7a5-4c18-8aab-0c7eec5279f2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:11.833749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24d2b6-d7a5-4c18-8aab-0c7eec5279f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:11.854780 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ff24d2b6-d7a5-4c18-8aab-0c7eec5279f2 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:20:11.856713 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ff24d2b6-d7a5-4c18-8aab-0c7eec5279f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:11.864960 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1108/2205] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:20:11.869478 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531f4f3-711a-4b0c-a23c-15db91364da9 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:11.880008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531f4f3-711a-4b0c-a23c-15db91364da9 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:11.886696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98ce2e-21b6-481d-a961-1a802932f0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:11.887931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98ce2e-21b6-481d-a961-1a802932f0b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:11.888213 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98ce2e-21b6-481d-a961-1a802932f0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:11.888487 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98ce2e-21b6-481d-a961-1a802932f0b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:11.889233 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98ce2e-21b6-481d-a961-1a802932f0b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:20:11.889539 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98ce2e-21b6-481d-a961-1a802932f0b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:11.891328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98ce2e-21b6-481d-a961-1a802932f0b6 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:11.907702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1098/2206] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:20:11.908953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24d2b6-d7a5-4c18-8aab-0c7eec5279f2 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:11.920118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24d2b6-d7a5-4c18-8aab-0c7eec5279f2 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:11.925036 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd29845-b4ee-4fcf-b617-0cc267495a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:11.925703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd29845-b4ee-4fcf-b617-0cc267495a2f admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:11.925985 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd29845-b4ee-4fcf-b617-0cc267495a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:11.926257 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd29845-b4ee-4fcf-b617-0cc267495a2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:11.926993 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd29845-b4ee-4fcf-b617-0cc267495a2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:11.928905 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd29845-b4ee-4fcf-b617-0cc267495a2f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:12.020022 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1109/2207] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:20:11 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:20:12.024400 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98ce2e-21b6-481d-a961-1a802932f0b6 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:12.036514 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98ce2e-21b6-481d-a961-1a802932f0b6 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:12.093542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a6780-abfa-41b9-8f87-76d35d54bfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:12.094211 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a6780-abfa-41b9-8f87-76d35d54bfaa admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:12.094530 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a6780-abfa-41b9-8f87-76d35d54bfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:12.094819 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a6780-abfa-41b9-8f87-76d35d54bfaa admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:12.095511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a6780-abfa-41b9-8f87-76d35d54bfaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:12.097437 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a6780-abfa-41b9-8f87-76d35d54bfaa admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:12.206978 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1099/2208] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:20:12 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:20:12.207074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd29845-b4ee-4fcf-b617-0cc267495a2f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:12.217823 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd29845-b4ee-4fcf-b617-0cc267495a2f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:12.222649 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb664405-e645-469b-9c38-9df28480ded1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:12.223320 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb664405-e645-469b-9c38-9df28480ded1 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:12.223604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb664405-e645-469b-9c38-9df28480ded1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:12.223881 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb664405-e645-469b-9c38-9df28480ded1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:12.224622 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb664405-e645-469b-9c38-9df28480ded1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:12.226427 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb664405-e645-469b-9c38-9df28480ded1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:12.237828 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1110/2209] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:20:12.241766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a6780-abfa-41b9-8f87-76d35d54bfaa admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:12.252806 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a6780-abfa-41b9-8f87-76d35d54bfaa admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:12.257556 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085330c4-dae2-43bb-8630-02dd098472dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:12.258225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085330c4-dae2-43bb-8630-02dd098472dd admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:12.258820 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085330c4-dae2-43bb-8630-02dd098472dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:12.259112 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085330c4-dae2-43bb-8630-02dd098472dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:12.259844 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085330c4-dae2-43bb-8630-02dd098472dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:12.261496 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085330c4-dae2-43bb-8630-02dd098472dd admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:12.274405 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1100/2210] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:20:12.277683 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb664405-e645-469b-9c38-9df28480ded1 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:12.288034 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb664405-e645-469b-9c38-9df28480ded1 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:12.292905 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0aee98-6974-4d0d-bb26-890ffa57bbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:12.293797 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0aee98-6974-4d0d-bb26-890ffa57bbb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:12.294083 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0aee98-6974-4d0d-bb26-890ffa57bbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:12.294361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0aee98-6974-4d0d-bb26-890ffa57bbb5 admin admin] PATH_INFO: `/v3/projects/bfe80bb94be043f4bfd04fb45d25472d/users/af80393967ee464a8c924dae5cff17be/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:12.309699 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0aee98-6974-4d0d-bb26-890ffa57bbb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfe80bb94be043f4bfd04fb45d25472d, user_id=af80393967ee464a8c924dae5cff17be, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:12.312416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0aee98-6974-4d0d-bb26-890ffa57bbb5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:12.384344 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1111/2211] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:20:12 2022] PUT /identity/v3/projects/bfe80bb94be043f4bfd04fb45d25472d/users/af80393967ee464a8c924dae5cff17be/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:20:12.394975 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dcd6a-ba56-4ba4-85cb-cff2e8395ec2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:12.395268 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dcd6a-ba56-4ba4-85cb-cff2e8395ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:12.395537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dcd6a-ba56-4ba4-85cb-cff2e8395ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:12.434470 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-367dcd6a-ba56-4ba4-85cb-cff2e8395ec2 None None] MFA Rules not processed for user `af80393967ee464a8c924dae5cff17be`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:20:12.435223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-367dcd6a-ba56-4ba4-85cb-cff2e8395ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:12.505821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:20:12.515281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1101/2212] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:20:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:20:12.529503 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0aee98-6974-4d0d-bb26-890ffa57bbb5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:12.531393 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ad0aee98-6974-4d0d-bb26-890ffa57bbb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:12.616778 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0aee98-6974-4d0d-bb26-890ffa57bbb5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:12.620500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f25d4f-f491-4399-a2c7-5940de0706eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:12.621215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f25d4f-f491-4399-a2c7-5940de0706eb service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:12.621426 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f25d4f-f491-4399-a2c7-5940de0706eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:12.621706 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f25d4f-f491-4399-a2c7-5940de0706eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:12.630280 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f25d4f-f491-4399-a2c7-5940de0706eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:12.635307 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f25d4f-f491-4399-a2c7-5940de0706eb service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:12.641140 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1112/2213] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:20:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:20:12.668825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367dcd6a-ba56-4ba4-85cb-cff2e8395ec2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:12.733841 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367dcd6a-ba56-4ba4-85cb-cff2e8395ec2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:12.739512 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e1b5a-180c-44e8-9933-f3581a6bae97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:12.740340 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e1b5a-180c-44e8-9933-f3581a6bae97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:12.740629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e1b5a-180c-44e8-9933-f3581a6bae97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:12.740912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e1b5a-180c-44e8-9933-f3581a6bae97 admin admin] PATH_INFO: `/v3/users/af80393967ee464a8c924dae5cff17be` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:12.742156 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588e1b5a-180c-44e8-9933-f3581a6bae97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af80393967ee464a8c924dae5cff17be)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:12.744295 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588e1b5a-180c-44e8-9933-f3581a6bae97 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:12.984177 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-588e1b5a-180c-44e8-9933-f3581a6bae97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af80393967ee464a8c924dae5cff17be'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:13.014946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-588e1b5a-180c-44e8-9933-f3581a6bae97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af80393967ee464a8c924dae5cff17be'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:13.101001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-588e1b5a-180c-44e8-9933-f3581a6bae97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af80393967ee464a8c924dae5cff17be'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:13.101803 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-588e1b5a-180c-44e8-9933-f3581a6bae97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af80393967ee464a8c924dae5cff17be'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:13.102642 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-588e1b5a-180c-44e8-9933-f3581a6bae97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af80393967ee464a8c924dae5cff17be'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:13.119314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1102/2214] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:20:12 2022] DELETE /identity/v3/users/af80393967ee464a8c924dae5cff17be => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:20:13.123774 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a06c3-792c-4d75-83bf-520506479171 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:13.125008 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a06c3-792c-4d75-83bf-520506479171 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:13.126021 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a06c3-792c-4d75-83bf-520506479171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:13.144652 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-042a06c3-792c-4d75-83bf-520506479171 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:20:13.146135 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-042a06c3-792c-4d75-83bf-520506479171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:13.239858 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1113/2215] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:20:13.256440 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e1b5a-180c-44e8-9933-f3581a6bae97 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:13.259152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-588e1b5a-180c-44e8-9933-f3581a6bae97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:13.326038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e1b5a-180c-44e8-9933-f3581a6bae97 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:13.329793 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b014d2c-4f71-4746-aa45-8c7ce227aabe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:13.330485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b014d2c-4f71-4746-aa45-8c7ce227aabe service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:13.330775 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b014d2c-4f71-4746-aa45-8c7ce227aabe service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:13.331082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b014d2c-4f71-4746-aa45-8c7ce227aabe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:13.341478 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b014d2c-4f71-4746-aa45-8c7ce227aabe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:13.343040 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b014d2c-4f71-4746-aa45-8c7ce227aabe service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:13.354078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1103/2216] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:20:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:20:14.051253 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a06c3-792c-4d75-83bf-520506479171 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:14.052758 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-042a06c3-792c-4d75-83bf-520506479171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:14.065267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a06c3-792c-4d75-83bf-520506479171 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:14.071371 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38837844-fa78-4c6e-80a8-f4603a25d517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:14.072489 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38837844-fa78-4c6e-80a8-f4603a25d517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:14.072895 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38837844-fa78-4c6e-80a8-f4603a25d517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:14.073242 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38837844-fa78-4c6e-80a8-f4603a25d517 admin admin] PATH_INFO: `/v3/projects/bfe80bb94be043f4bfd04fb45d25472d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:14.075913 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38837844-fa78-4c6e-80a8-f4603a25d517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfe80bb94be043f4bfd04fb45d25472d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:14.078109 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38837844-fa78-4c6e-80a8-f4603a25d517 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:14.105606 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:20:14.195858 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-38837844-fa78-4c6e-80a8-f4603a25d517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfe80bb94be043f4bfd04fb45d25472d', 'request_id': 'req-38837844-fa78-4c6e-80a8-f4603a25d517'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:14.203159 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-38837844-fa78-4c6e-80a8-f4603a25d517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfe80bb94be043f4bfd04fb45d25472d', 'request_id': 'req-38837844-fa78-4c6e-80a8-f4603a25d517'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:14.210441 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-38837844-fa78-4c6e-80a8-f4603a25d517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfe80bb94be043f4bfd04fb45d25472d', 'request_id': 'req-38837844-fa78-4c6e-80a8-f4603a25d517'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:14.313866 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-38837844-fa78-4c6e-80a8-f4603a25d517 admin admin] The token cache is being invalidate because project bfe80bb94be043f4bfd04fb45d25472d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:20:14.315136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-38837844-fa78-4c6e-80a8-f4603a25d517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:14.317352 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-38837844-fa78-4c6e-80a8-f4603a25d517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:20:14.319275 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1114/2217] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:20:14 2022] DELETE /identity/v3/projects/bfe80bb94be043f4bfd04fb45d25472d => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:20:20.922183 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b014d2c-4f71-4746-aa45-8c7ce227aabe service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:20.928486 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5b014d2c-4f71-4746-aa45-8c7ce227aabe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:20.957172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:20:21.043659 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b014d2c-4f71-4746-aa45-8c7ce227aabe service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:21.132094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1870e27-7134-4410-80b5-0acd32213d81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7662e61619d44c92af3ffacbacba3913', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:21.132792 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1870e27-7134-4410-80b5-0acd32213d81 service placement] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:21.133080 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1870e27-7134-4410-80b5-0acd32213d81 service placement] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:21.133367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1870e27-7134-4410-80b5-0acd32213d81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:21.135518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d1870e27-7134-4410-80b5-0acd32213d81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:21.160921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1870e27-7134-4410-80b5-0acd32213d81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:21.162595 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1870e27-7134-4410-80b5-0acd32213d81 service placement] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:21.174403 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1104/2218] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:20:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:20:22.447294 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc2a71-278d-4a00-b958-601a8f7e4b8e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:22.447774 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc2a71-278d-4a00-b958-601a8f7e4b8e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:22.447949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc2a71-278d-4a00-b958-601a8f7e4b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:22.466865 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f0bc2a71-278d-4a00-b958-601a8f7e4b8e None None] MFA Rules not processed for user `8c6ef9f5f6ac4b6289aa1aa9e16ce4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:20:22.468078 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f0bc2a71-278d-4a00-b958-601a8f7e4b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:22.536693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1115/2219] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:20:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:20:22.591279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c2927-e2af-4959-9f66-9e7fffad2352 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:22.593101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c2927-e2af-4959-9f66-9e7fffad2352 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:22.593519 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5c2927-e2af-4959-9f66-9e7fffad2352 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:22.595060 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1105/2220] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jun 24 04:20:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:20:22.601431 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6d870-0fcd-4efd-b3dc-256af51d2cc7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:22.601704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6d870-0fcd-4efd-b3dc-256af51d2cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:22.602020 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6d870-0fcd-4efd-b3dc-256af51d2cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:22.661911 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-80f6d870-0fcd-4efd-b3dc-256af51d2cc7 None None] MFA Rules not processed for user `80b7b3ec60db4de19bbf7d08f111f470`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:20:22.663174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-80f6d870-0fcd-4efd-b3dc-256af51d2cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:22.764348 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1116/2221] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jun 24 04:20:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:20:22.776092 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c2927-e2af-4959-9f66-9e7fffad2352 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:22.779409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6e5c2927-e2af-4959-9f66-9e7fffad2352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:22.901696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5c2927-e2af-4959-9f66-9e7fffad2352 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:22.906301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e91ecc-e9d6-4db6-bd3c-07b0ae4ca2f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61459e15d327405b88a2fdfc1fc0fecd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:22.906989 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e91ecc-e9d6-4db6-bd3c-07b0ae4ca2f5 service glance] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:22.907842 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e91ecc-e9d6-4db6-bd3c-07b0ae4ca2f5 service glance] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:22.907842 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e91ecc-e9d6-4db6-bd3c-07b0ae4ca2f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:22.918899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e91ecc-e9d6-4db6-bd3c-07b0ae4ca2f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:22.920450 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e91ecc-e9d6-4db6-bd3c-07b0ae4ca2f5 service glance] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:22.932351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1106/2222] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:20:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:20:23.148382 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93748c22-dc0b-4288-a19a-9ef2dc250de1 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:23.148738 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93748c22-dc0b-4288-a19a-9ef2dc250de1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:23.149087 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93748c22-dc0b-4288-a19a-9ef2dc250de1 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:23.150476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1117/2223] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jun 24 04:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 04:20:23.156302 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32060633-0694-4883-af87-2586a603b8f8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:23.156685 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32060633-0694-4883-af87-2586a603b8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:23.157035 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32060633-0694-4883-af87-2586a603b8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:23.176934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-32060633-0694-4883-af87-2586a603b8f8 None None] MFA Rules not processed for user `fd98b54f587f4b42b4dd0a058b30c684`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:20:23.192985 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-32060633-0694-4883-af87-2586a603b8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:23.214777 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:20:23.281696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1107/2224] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jun 24 04:20:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:20:23.293966 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93748c22-dc0b-4288-a19a-9ef2dc250de1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:23.296242 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-93748c22-dc0b-4288-a19a-9ef2dc250de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:23.311810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93748c22-dc0b-4288-a19a-9ef2dc250de1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:23.315512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd14e3f-bf76-460b-bff8-58d1574ae76d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:23.316476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd14e3f-bf76-460b-bff8-58d1574ae76d service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:23.316840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd14e3f-bf76-460b-bff8-58d1574ae76d service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:23.317180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd14e3f-bf76-460b-bff8-58d1574ae76d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:23.321576 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:20:23.326619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd14e3f-bf76-460b-bff8-58d1574ae76d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:23.328182 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd14e3f-bf76-460b-bff8-58d1574ae76d service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:23.342427 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1118/2225] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:20:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:20:23.451356 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9704b-9c51-47c2-a503-c402e9e25554 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:23.451961 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9704b-9c51-47c2-a503-c402e9e25554 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:23.456184 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b9704b-9c51-47c2-a503-c402e9e25554 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:23.456184 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1108/2226] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jun 24 04:20:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:20:23.459807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd79e64-0a8e-4ae0-b7b3-7b6979ce4dd5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:23.460280 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd79e64-0a8e-4ae0-b7b3-7b6979ce4dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:23.460361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd79e64-0a8e-4ae0-b7b3-7b6979ce4dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:23.481974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-8dd79e64-0a8e-4ae0-b7b3-7b6979ce4dd5 None None] MFA Rules not processed for user `80b7b3ec60db4de19bbf7d08f111f470`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:20:23.483294 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8dd79e64-0a8e-4ae0-b7b3-7b6979ce4dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:23.492604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1119/2227] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jun 24 04:20:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:20:23.503761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9704b-9c51-47c2-a503-c402e9e25554 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:23.515475 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b9704b-9c51-47c2-a503-c402e9e25554 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:23.519391 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc2f63-385a-45d2-916f-b6222cfd7bf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:23.520051 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc2f63-385a-45d2-916f-b6222cfd7bf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:23.520307 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc2f63-385a-45d2-916f-b6222cfd7bf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:23.520608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc2f63-385a-45d2-916f-b6222cfd7bf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:23.534391 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc2f63-385a-45d2-916f-b6222cfd7bf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:23.535854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc2f63-385a-45d2-916f-b6222cfd7bf7 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:23.549103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1109/2228] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:20:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:20:33.102807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d33d1-ebaf-404f-aed9-0ad7745c63a8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:33.103301 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d33d1-ebaf-404f-aed9-0ad7745c63a8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:33.103405 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d33d1-ebaf-404f-aed9-0ad7745c63a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:33.161567 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3d7d33d1-ebaf-404f-aed9-0ad7745c63a8 None None] MFA Rules not processed for user `98f58f91d17949198e66883d234a9e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:20:33.162744 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3d7d33d1-ebaf-404f-aed9-0ad7745c63a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:33.245643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1120/2229] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jun 24 04:20:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:20:33.258318 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc2f63-385a-45d2-916f-b6222cfd7bf7 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:33.259922 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-42cc2f63-385a-45d2-916f-b6222cfd7bf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:33.370633 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc2f63-385a-45d2-916f-b6222cfd7bf7 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:33.376496 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835385ff-f19d-47d6-b98b-e486b5892a1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8e1f1e3c7e42cc9fa6f82aaac55129', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:33.377240 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835385ff-f19d-47d6-b98b-e486b5892a1d service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:33.377685 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835385ff-f19d-47d6-b98b-e486b5892a1d service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:33.378036 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835385ff-f19d-47d6-b98b-e486b5892a1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:33.387784 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835385ff-f19d-47d6-b98b-e486b5892a1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:33.392448 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835385ff-f19d-47d6-b98b-e486b5892a1d service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:33.392586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d33d1-ebaf-404f-aed9-0ad7745c63a8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:33.401076 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1110/2230] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jun 24 04:20:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 04:20:33.410535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d33d1-ebaf-404f-aed9-0ad7745c63a8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:33.418011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d5438-48c0-4d70-a107-0bd1ec202653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:33.419445 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d5438-48c0-4d70-a107-0bd1ec202653 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:33.419935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d5438-48c0-4d70-a107-0bd1ec202653 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:33.420218 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d5438-48c0-4d70-a107-0bd1ec202653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:33.422696 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fc7d5438-48c0-4d70-a107-0bd1ec202653 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:33.502734 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d5438-48c0-4d70-a107-0bd1ec202653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:33.504201 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d5438-48c0-4d70-a107-0bd1ec202653 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:33.515589 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1121/2231] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:20:33 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:20:43.396069 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=63 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:20:43.509968 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=76 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:20:48.003629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835385ff-f19d-47d6-b98b-e486b5892a1d service swift] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:20:48.005116 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-835385ff-f19d-47d6-b98b-e486b5892a1d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:48.018231 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835385ff-f19d-47d6-b98b-e486b5892a1d service swift] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:20:48.021936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d32edd-71e1-49d0-8bad-a9385967d34f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:20:48.022613 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d32edd-71e1-49d0-8bad-a9385967d34f service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:20:48.023165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d32edd-71e1-49d0-8bad-a9385967d34f service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:20:48.023441 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d32edd-71e1-49d0-8bad-a9385967d34f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:20:48.025214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-10d32edd-71e1-49d0-8bad-a9385967d34f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:20:48.088149 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d32edd-71e1-49d0-8bad-a9385967d34f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:20:48.089579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d32edd-71e1-49d0-8bad-a9385967d34f service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:20:48.099731 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1111/2232] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:20:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:20:58.095427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=30 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:21:01.249430 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df0136-56eb-42e8-94d1-a1fd2dcad3fe None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:01.249930 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df0136-56eb-42e8-94d1-a1fd2dcad3fe None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:01.250023 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df0136-56eb-42e8-94d1-a1fd2dcad3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:01.270493 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f7df0136-56eb-42e8-94d1-a1fd2dcad3fe None None] MFA Rules not processed for user `98f58f91d17949198e66883d234a9e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:21:01.271914 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f7df0136-56eb-42e8-94d1-a1fd2dcad3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:21:01.279882 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1122/2233] 10.222.0.16 () {58 vars in 967 bytes} [Fri Jun 24 04:21:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:21:01.300842 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d32edd-71e1-49d0-8bad-a9385967d34f service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:21:01.312953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d32edd-71e1-49d0-8bad-a9385967d34f service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:21:01.317860 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adbbb3-2eb5-4006-9f87-056a51a5c62d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8e1f1e3c7e42cc9fa6f82aaac55129', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:21:01.318565 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adbbb3-2eb5-4006-9f87-056a51a5c62d service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:01.318848 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adbbb3-2eb5-4006-9f87-056a51a5c62d service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:01.319123 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adbbb3-2eb5-4006-9f87-056a51a5c62d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:01.327967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adbbb3-2eb5-4006-9f87-056a51a5c62d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:21:01.329539 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adbbb3-2eb5-4006-9f87-056a51a5c62d service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:21:01.337504 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1112/2234] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jun 24 04:21:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 04:21:05.669257 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a2d6a-348d-4c53-bee3-03214f164a8c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:05.669779 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a2d6a-348d-4c53-bee3-03214f164a8c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:05.669916 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85a2d6a-348d-4c53-bee3-03214f164a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:05.689868 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d85a2d6a-348d-4c53-bee3-03214f164a8c None None] MFA Rules not processed for user `98f58f91d17949198e66883d234a9e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:21:05.691122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d85a2d6a-348d-4c53-bee3-03214f164a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:21:05.698545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1123/2235] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jun 24 04:21:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:21:05.718455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adbbb3-2eb5-4006-9f87-056a51a5c62d service swift] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:21:05.729199 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adbbb3-2eb5-4006-9f87-056a51a5c62d service swift] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:21:05.734984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b0b87-751e-4083-bc63-3adfedd167b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8e1f1e3c7e42cc9fa6f82aaac55129', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:21:05.737628 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b0b87-751e-4083-bc63-3adfedd167b9 service swift] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:05.737628 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b0b87-751e-4083-bc63-3adfedd167b9 service swift] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:05.737628 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b0b87-751e-4083-bc63-3adfedd167b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:05.744814 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b0b87-751e-4083-bc63-3adfedd167b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:21:05.747035 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b0b87-751e-4083-bc63-3adfedd167b9 service swift] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:21:05.755332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1113/2236] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jun 24 04:21:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 24 04:21:15.031382 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c348ab-f864-45d7-b719-767cdfe473e1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:15.031981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c348ab-f864-45d7-b719-767cdfe473e1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:15.031981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c348ab-f864-45d7-b719-767cdfe473e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:15.051189 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-41c348ab-f864-45d7-b719-767cdfe473e1 None None] MFA Rules not processed for user `114d227e9645409e9082cf9b09b05c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:21:15.052400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-41c348ab-f864-45d7-b719-767cdfe473e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:21:15.135154 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1124/2237] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:21:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:21:15.751948 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=12 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:21:22.913485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa66cf-9e71-40de-9bad-26fcf2218b81 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:22.914066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa66cf-9e71-40de-9bad-26fcf2218b81 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:22.914066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa66cf-9e71-40de-9bad-26fcf2218b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:22.931619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-6faa66cf-9e71-40de-9bad-26fcf2218b81 None None] MFA Rules not processed for user `98f58f91d17949198e66883d234a9e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:21:22.932813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6faa66cf-9e71-40de-9bad-26fcf2218b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:21:22.943989 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1114/2238] 10.222.0.16 () {58 vars in 967 bytes} [Fri Jun 24 04:21:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:21:22.956548 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c348ab-f864-45d7-b719-767cdfe473e1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:21:22.968490 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c348ab-f864-45d7-b719-767cdfe473e1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:21:22.973459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3176f-62eb-4fd9-ab05-e78cd132e3ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8e1f1e3c7e42cc9fa6f82aaac55129', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:21:22.974124 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3176f-62eb-4fd9-ab05-e78cd132e3ae service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:22.974461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3176f-62eb-4fd9-ab05-e78cd132e3ae service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:22.974910 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3176f-62eb-4fd9-ab05-e78cd132e3ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:22.983324 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3176f-62eb-4fd9-ab05-e78cd132e3ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:21:22.984906 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3176f-62eb-4fd9-ab05-e78cd132e3ae service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:21:22.992885 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1125/2239] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jun 24 04:21:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 04:21:26.121868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32e42c-c9ed-4699-95b9-055b70761c19 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:26.122473 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32e42c-c9ed-4699-95b9-055b70761c19 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:26.122473 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32e42c-c9ed-4699-95b9-055b70761c19 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:26.124468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1115/2240] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jun 24 04:21:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:21:26.135635 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5a9f7-475c-4c78-a604-efd2d8180bec None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:26.136096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5a9f7-475c-4c78-a604-efd2d8180bec None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:26.136315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5a9f7-475c-4c78-a604-efd2d8180bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:26.141866 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=38 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:21:26.179354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9cf5a9f7-475c-4c78-a604-efd2d8180bec None None] MFA Rules not processed for user `7662e61619d44c92af3ffacbacba3913`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:21:26.181527 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9cf5a9f7-475c-4c78-a604-efd2d8180bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:21:26.192705 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1126/2241] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jun 24 04:21:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:21:26.204429 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32e42c-c9ed-4699-95b9-055b70761c19 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:21:26.206113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2d32e42c-c9ed-4699-95b9-055b70761c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:21:26.222675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32e42c-c9ed-4699-95b9-055b70761c19 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:21:26.227722 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9bc95f-958e-45e5-8809-b3af2a75ae63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7662e61619d44c92af3ffacbacba3913', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:21:26.232635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bc95f-958e-45e5-8809-b3af2a75ae63 service placement] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:26.232635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bc95f-958e-45e5-8809-b3af2a75ae63 service placement] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:26.232635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bc95f-958e-45e5-8809-b3af2a75ae63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:26.238290 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9bc95f-958e-45e5-8809-b3af2a75ae63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:21:26.239882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9bc95f-958e-45e5-8809-b3af2a75ae63 service placement] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:21:26.250193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1116/2242] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:21:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:21:26.296668 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf5a9f7-475c-4c78-a604-efd2d8180bec None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:21:26.298243 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9cf5a9f7-475c-4c78-a604-efd2d8180bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:21:26.362695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf5a9f7-475c-4c78-a604-efd2d8180bec None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:21:26.367626 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e0cb95-a44b-4eea-bbd9-3e026ea52b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:21:26.368519 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0cb95-a44b-4eea-bbd9-3e026ea52b52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:26.368937 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0cb95-a44b-4eea-bbd9-3e026ea52b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:26.369706 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0cb95-a44b-4eea-bbd9-3e026ea52b52 admin admin] PATH_INFO: `/v3/users/8c6ef9f5f6ac4b6289aa1aa9e16ce4bb` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:26.370948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e0cb95-a44b-4eea-bbd9-3e026ea52b52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c6ef9f5f6ac4b6289aa1aa9e16ce4bb)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:21:26.372923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e0cb95-a44b-4eea-bbd9-3e026ea52b52 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:21:26.582461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a9e0cb95-a44b-4eea-bbd9-3e026ea52b52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c6ef9f5f6ac4b6289aa1aa9e16ce4bb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:26.594607 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a9e0cb95-a44b-4eea-bbd9-3e026ea52b52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c6ef9f5f6ac4b6289aa1aa9e16ce4bb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:26.595300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a9e0cb95-a44b-4eea-bbd9-3e026ea52b52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c6ef9f5f6ac4b6289aa1aa9e16ce4bb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:26.595914 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a9e0cb95-a44b-4eea-bbd9-3e026ea52b52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c6ef9f5f6ac4b6289aa1aa9e16ce4bb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:26.603252 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a9e0cb95-a44b-4eea-bbd9-3e026ea52b52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c6ef9f5f6ac4b6289aa1aa9e16ce4bb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:26.688590 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1127/2243] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:21:26 2022] DELETE /identity/v3/users/8c6ef9f5f6ac4b6289aa1aa9e16ce4bb => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:21:27.047188 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9bc95f-958e-45e5-8809-b3af2a75ae63 service placement] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:21:27.060516 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9bc95f-958e-45e5-8809-b3af2a75ae63 service placement] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:21:27.121522 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c144980f-6e5b-4b1a-845d-26acb6278a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:21:27.122415 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144980f-6e5b-4b1a-845d-26acb6278a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:27.123242 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144980f-6e5b-4b1a-845d-26acb6278a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:27.123242 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c144980f-6e5b-4b1a-845d-26acb6278a49 admin admin] PATH_INFO: `/v3/projects/46222aa2184c4a6d82b65d3c703c8bd7` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:27.125067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144980f-6e5b-4b1a-845d-26acb6278a49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46222aa2184c4a6d82b65d3c703c8bd7)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:21:27.127042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c144980f-6e5b-4b1a-845d-26acb6278a49 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:21:27.241340 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c144980f-6e5b-4b1a-845d-26acb6278a49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46222aa2184c4a6d82b65d3c703c8bd7', 'request_id': 'req-c144980f-6e5b-4b1a-845d-26acb6278a49'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:27.305351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c144980f-6e5b-4b1a-845d-26acb6278a49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46222aa2184c4a6d82b65d3c703c8bd7', 'request_id': 'req-c144980f-6e5b-4b1a-845d-26acb6278a49'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:27.312184 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c144980f-6e5b-4b1a-845d-26acb6278a49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46222aa2184c4a6d82b65d3c703c8bd7', 'request_id': 'req-c144980f-6e5b-4b1a-845d-26acb6278a49'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:27.323541 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c144980f-6e5b-4b1a-845d-26acb6278a49 admin admin] The token cache is being invalidate because project 46222aa2184c4a6d82b65d3c703c8bd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:21:27.323837 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c144980f-6e5b-4b1a-845d-26acb6278a49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:27.324493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c144980f-6e5b-4b1a-845d-26acb6278a49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:27.326172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1117/2244] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:21:27 2022] DELETE /identity/v3/projects/46222aa2184c4a6d82b65d3c703c8bd7 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:21:27.332043 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176448d0-8712-47ea-8872-bcd1444a5d22 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:27.332599 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176448d0-8712-47ea-8872-bcd1444a5d22 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:27.332852 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176448d0-8712-47ea-8872-bcd1444a5d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:27.351603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-176448d0-8712-47ea-8872-bcd1444a5d22 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:21:27.352805 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-176448d0-8712-47ea-8872-bcd1444a5d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:21:27.423386 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1128/2245] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:21:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:21:27.433269 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d6f23-53f0-474b-b9d5-66b231f3e128 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:27.433556 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d6f23-53f0-474b-b9d5-66b231f3e128 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:27.433836 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d6f23-53f0-474b-b9d5-66b231f3e128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:27.453944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-cb4d6f23-53f0-474b-b9d5-66b231f3e128 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:21:27.455196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-cb4d6f23-53f0-474b-b9d5-66b231f3e128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:21:27.463035 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1118/2246] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:21:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:21:27.467207 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176448d0-8712-47ea-8872-bcd1444a5d22 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:21:27.472094 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:21:27.477431 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176448d0-8712-47ea-8872-bcd1444a5d22 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:21:27.482011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530ba87-1f08-4789-8dd3-8a72eaf3970d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:21:27.486476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530ba87-1f08-4789-8dd3-8a72eaf3970d admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:27.486476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530ba87-1f08-4789-8dd3-8a72eaf3970d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:27.486476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530ba87-1f08-4789-8dd3-8a72eaf3970d admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:27.486476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530ba87-1f08-4789-8dd3-8a72eaf3970d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:21:27.486476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530ba87-1f08-4789-8dd3-8a72eaf3970d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:21:27.486476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530ba87-1f08-4789-8dd3-8a72eaf3970d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:21:27.498123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1129/2247] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:21:27.502915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d6f23-53f0-474b-b9d5-66b231f3e128 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:21:27.512062 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d6f23-53f0-474b-b9d5-66b231f3e128 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:21:27.517585 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ede070-77d2-465c-9bdc-96359d545b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:21:27.518787 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ede070-77d2-465c-9bdc-96359d545b49 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:27.519072 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ede070-77d2-465c-9bdc-96359d545b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:27.519349 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ede070-77d2-465c-9bdc-96359d545b49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:27.520050 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ede070-77d2-465c-9bdc-96359d545b49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:21:27.521991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ede070-77d2-465c-9bdc-96359d545b49 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:21:27.632877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1119/2248] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:21:27 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:21:27.636742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530ba87-1f08-4789-8dd3-8a72eaf3970d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:21:27.647753 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530ba87-1f08-4789-8dd3-8a72eaf3970d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:21:27.708566 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699331a-af89-4776-a104-40d7cda5a144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:21:27.709264 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699331a-af89-4776-a104-40d7cda5a144 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:27.709554 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699331a-af89-4776-a104-40d7cda5a144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:27.709834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699331a-af89-4776-a104-40d7cda5a144 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:27.713863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4699331a-af89-4776-a104-40d7cda5a144 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:21:27.713863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4699331a-af89-4776-a104-40d7cda5a144 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:21:27.800101 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1130/2249] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:21:27 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:21:27.801517 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ede070-77d2-465c-9bdc-96359d545b49 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:21:27.812197 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ede070-77d2-465c-9bdc-96359d545b49 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:21:27.816902 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af83c9-5003-445f-b1e1-4240923914c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:21:27.817568 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af83c9-5003-445f-b1e1-4240923914c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:27.817850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af83c9-5003-445f-b1e1-4240923914c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:27.818129 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af83c9-5003-445f-b1e1-4240923914c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:27.818897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af83c9-5003-445f-b1e1-4240923914c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:21:27.820528 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3af83c9-5003-445f-b1e1-4240923914c1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:21:27.833122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1120/2250] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:21:27.837278 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699331a-af89-4776-a104-40d7cda5a144 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:21:27.849283 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699331a-af89-4776-a104-40d7cda5a144 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:21:27.854144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f799f8-18e0-412a-84da-7e4bc59551ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:21:27.854849 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f799f8-18e0-412a-84da-7e4bc59551ee admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:27.855139 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f799f8-18e0-412a-84da-7e4bc59551ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:27.855418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f799f8-18e0-412a-84da-7e4bc59551ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:27.856149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f799f8-18e0-412a-84da-7e4bc59551ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:21:27.857944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f799f8-18e0-412a-84da-7e4bc59551ee admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:21:27.868125 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1131/2251] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:21:27.872292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af83c9-5003-445f-b1e1-4240923914c1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:21:27.882971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3af83c9-5003-445f-b1e1-4240923914c1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:21:27.887639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b67aa-d1a8-4cff-b4ef-0c4ba5b30208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:21:27.888546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b67aa-d1a8-4cff-b4ef-0c4ba5b30208 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:27.888830 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b67aa-d1a8-4cff-b4ef-0c4ba5b30208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:27.889660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186b67aa-d1a8-4cff-b4ef-0c4ba5b30208 admin admin] PATH_INFO: `/v3/projects/89295eb3353747889b6e12bc48ec083d/users/28d253fc296a4497a75d3d4c2fbc4dde/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:27.904891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b67aa-d1a8-4cff-b4ef-0c4ba5b30208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89295eb3353747889b6e12bc48ec083d, user_id=28d253fc296a4497a75d3d4c2fbc4dde, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:21:27.907620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186b67aa-d1a8-4cff-b4ef-0c4ba5b30208 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:21:27.970657 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1121/2252] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:21:27 2022] PUT /identity/v3/projects/89295eb3353747889b6e12bc48ec083d/users/28d253fc296a4497a75d3d4c2fbc4dde/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:21:27.980886 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cad1c0-a585-44ea-ba47-895acd92fc55 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:27.982594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cad1c0-a585-44ea-ba47-895acd92fc55 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:27.982594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cad1c0-a585-44ea-ba47-895acd92fc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:28.012525 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-93cad1c0-a585-44ea-ba47-895acd92fc55 None None] MFA Rules not processed for user `28d253fc296a4497a75d3d4c2fbc4dde`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:21:28.013735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-93cad1c0-a585-44ea-ba47-895acd92fc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:21:28.081564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1132/2253] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:21:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:21:28.093597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b67aa-d1a8-4cff-b4ef-0c4ba5b30208 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:21:28.095411 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-186b67aa-d1a8-4cff-b4ef-0c4ba5b30208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:21:28.106513 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:21:28.157866 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186b67aa-d1a8-4cff-b4ef-0c4ba5b30208 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:21:28.161718 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a77c6-0bc8-4d18-b894-dae266039624 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:21:28.162466 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a77c6-0bc8-4d18-b894-dae266039624 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:28.162739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a77c6-0bc8-4d18-b894-dae266039624 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:28.163019 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a77c6-0bc8-4d18-b894-dae266039624 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:28.177512 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a77c6-0bc8-4d18-b894-dae266039624 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:21:28.177512 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a77c6-0bc8-4d18-b894-dae266039624 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:21:28.189067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1122/2254] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:21:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:21:38.078655 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:21:38.225532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=23 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:21:39.830005 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cad1c0-a585-44ea-ba47-895acd92fc55 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:21:39.892972 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cad1c0-a585-44ea-ba47-895acd92fc55 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:21:39.897412 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bb0e9-ab13-4c5d-b953-3e29a678d18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:21:39.898561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bb0e9-ab13-4c5d-b953-3e29a678d18c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:39.898846 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bb0e9-ab13-4c5d-b953-3e29a678d18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:39.899120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bb0e9-ab13-4c5d-b953-3e29a678d18c admin admin] PATH_INFO: `/v3/users/28d253fc296a4497a75d3d4c2fbc4dde` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:39.900295 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999bb0e9-ab13-4c5d-b953-3e29a678d18c admin admin] RBAC: Authorizing `identity:delete_user(user_id=28d253fc296a4497a75d3d4c2fbc4dde)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:21:39.902222 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999bb0e9-ab13-4c5d-b953-3e29a678d18c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:21:40.081204 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-999bb0e9-ab13-4c5d-b953-3e29a678d18c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28d253fc296a4497a75d3d4c2fbc4dde'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:40.093388 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-999bb0e9-ab13-4c5d-b953-3e29a678d18c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28d253fc296a4497a75d3d4c2fbc4dde'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:40.094064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-999bb0e9-ab13-4c5d-b953-3e29a678d18c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28d253fc296a4497a75d3d4c2fbc4dde'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:40.094687 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-999bb0e9-ab13-4c5d-b953-3e29a678d18c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28d253fc296a4497a75d3d4c2fbc4dde'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:40.109538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-999bb0e9-ab13-4c5d-b953-3e29a678d18c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28d253fc296a4497a75d3d4c2fbc4dde'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:40.184927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1133/2255] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:21:39 2022] DELETE /identity/v3/users/28d253fc296a4497a75d3d4c2fbc4dde => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:21:40.189631 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c024ef-bace-4fbf-b77c-ea8211df355a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:40.190185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c024ef-bace-4fbf-b77c-ea8211df355a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:40.190292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c024ef-bace-4fbf-b77c-ea8211df355a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:40.207600 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-96c024ef-bace-4fbf-b77c-ea8211df355a None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:21:40.208810 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-96c024ef-bace-4fbf-b77c-ea8211df355a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:21:40.268134 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1123/2256] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:21:40.277973 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bb0e9-ab13-4c5d-b953-3e29a678d18c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:21:40.279424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-999bb0e9-ab13-4c5d-b953-3e29a678d18c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:21:40.337894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bb0e9-ab13-4c5d-b953-3e29a678d18c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:21:40.341520 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1338d58-611c-487a-a84e-9eed782fecdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:21:40.342810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1338d58-611c-487a-a84e-9eed782fecdb service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:40.343266 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1338d58-611c-487a-a84e-9eed782fecdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:40.343475 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1338d58-611c-487a-a84e-9eed782fecdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:40.353056 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1338d58-611c-487a-a84e-9eed782fecdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:21:40.354540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1338d58-611c-487a-a84e-9eed782fecdb service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:21:40.364780 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1134/2257] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:21:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:21:40.821217 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c024ef-bace-4fbf-b77c-ea8211df355a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:21:40.822669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-96c024ef-bace-4fbf-b77c-ea8211df355a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:21:40.839003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c024ef-bace-4fbf-b77c-ea8211df355a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:21:40.843749 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16da00-c62d-4398-99a1-d88d7aa3870a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:21:40.844618 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16da00-c62d-4398-99a1-d88d7aa3870a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:40.844900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16da00-c62d-4398-99a1-d88d7aa3870a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:40.845175 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16da00-c62d-4398-99a1-d88d7aa3870a admin admin] PATH_INFO: `/v3/projects/89295eb3353747889b6e12bc48ec083d` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:40.846550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16da00-c62d-4398-99a1-d88d7aa3870a admin admin] RBAC: Authorizing `identity:delete_project(project_id=89295eb3353747889b6e12bc48ec083d)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:21:40.849687 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16da00-c62d-4398-99a1-d88d7aa3870a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:21:40.933443 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef16da00-c62d-4398-99a1-d88d7aa3870a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89295eb3353747889b6e12bc48ec083d', 'request_id': 'req-ef16da00-c62d-4398-99a1-d88d7aa3870a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:41.001899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef16da00-c62d-4398-99a1-d88d7aa3870a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89295eb3353747889b6e12bc48ec083d', 'request_id': 'req-ef16da00-c62d-4398-99a1-d88d7aa3870a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:41.007744 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef16da00-c62d-4398-99a1-d88d7aa3870a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89295eb3353747889b6e12bc48ec083d', 'request_id': 'req-ef16da00-c62d-4398-99a1-d88d7aa3870a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:41.019636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef16da00-c62d-4398-99a1-d88d7aa3870a admin admin] The token cache is being invalidate because project 89295eb3353747889b6e12bc48ec083d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:21:41.019927 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef16da00-c62d-4398-99a1-d88d7aa3870a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:41.020570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef16da00-c62d-4398-99a1-d88d7aa3870a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:21:41.022120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1124/2258] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:21:40 2022] DELETE /identity/v3/projects/89295eb3353747889b6e12bc48ec083d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:21:50.360386 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:21:51.016690 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:21:54.880239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5de1b8-1502-4931-b984-db4a780ce419 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:54.880802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5de1b8-1502-4931-b984-db4a780ce419 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:54.880913 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5de1b8-1502-4931-b984-db4a780ce419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:54.899703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2b5de1b8-1502-4931-b984-db4a780ce419 None None] MFA Rules not processed for user `b158078d039a4979a9c0cf2be26c4b14`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:21:54.900920 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2b5de1b8-1502-4931-b984-db4a780ce419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:21:54.957767 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1135/2259] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:21:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:21:58.712870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db755265-8b11-4317-9740-c802aa020e84 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:21:58.712870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db755265-8b11-4317-9740-c802aa020e84 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:21:58.712870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db755265-8b11-4317-9740-c802aa020e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:21:58.727742 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-db755265-8b11-4317-9740-c802aa020e84 None None] MFA Rules not processed for user `9ffa029a3b1e4919b00e58a9737816d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:21:58.729051 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-db755265-8b11-4317-9740-c802aa020e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:21:58.784328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1125/2260] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:21:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:22:04.950275 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=25 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:22:08.777016 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=25 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:22:15.684113 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5de1b8-1502-4931-b984-db4a780ce419 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:15.686156 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2b5de1b8-1502-4931-b984-db4a780ce419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:15.741655 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5de1b8-1502-4931-b984-db4a780ce419 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:15.745244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841e977-75f4-40ec-9cb7-abdb564daa8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:15.745883 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841e977-75f4-40ec-9cb7-abdb564daa8e service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:15.746168 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841e977-75f4-40ec-9cb7-abdb564daa8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:15.746475 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841e977-75f4-40ec-9cb7-abdb564daa8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:15.749363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4841e977-75f4-40ec-9cb7-abdb564daa8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:15.818579 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4841e977-75f4-40ec-9cb7-abdb564daa8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:15.820030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4841e977-75f4-40ec-9cb7-abdb564daa8e service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:15.831286 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1136/2261] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:22:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:22:20.809624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db755265-8b11-4317-9740-c802aa020e84 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:20.811919 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-db755265-8b11-4317-9740-c802aa020e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:20.872171 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db755265-8b11-4317-9740-c802aa020e84 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:20.876608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de224dc9-4d29-476a-8839-eff44dad6350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:20.877261 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de224dc9-4d29-476a-8839-eff44dad6350 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:20.877543 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de224dc9-4d29-476a-8839-eff44dad6350 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:20.877821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de224dc9-4d29-476a-8839-eff44dad6350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:20.879641 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-de224dc9-4d29-476a-8839-eff44dad6350 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:20.893606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de224dc9-4d29-476a-8839-eff44dad6350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:20.895117 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de224dc9-4d29-476a-8839-eff44dad6350 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:20.907254 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1126/2262] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:22:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:22:25.826019 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=54 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:22:30.217091 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf3b1f-6f33-4899-a4e0-4d3da6d9083a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:30.217091 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf3b1f-6f33-4899-a4e0-4d3da6d9083a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:30.217705 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf3b1f-6f33-4899-a4e0-4d3da6d9083a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:30.242153 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-bcbf3b1f-6f33-4899-a4e0-4d3da6d9083a None None] MFA Rules not processed for user `066d1521db464b4996f6daf6548dcc38`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:22:30.243465 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-bcbf3b1f-6f33-4899-a4e0-4d3da6d9083a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:30.333948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1137/2263] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:22:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:22:30.901489 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=26 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:22:31.926412 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de224dc9-4d29-476a-8839-eff44dad6350 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:31.928591 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-de224dc9-4d29-476a-8839-eff44dad6350 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:31.946844 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de224dc9-4d29-476a-8839-eff44dad6350 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:31.955715 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7234301-ccba-460c-b072-a45dd510116b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:31.956419 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7234301-ccba-460c-b072-a45dd510116b service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:31.956912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7234301-ccba-460c-b072-a45dd510116b service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:31.957372 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7234301-ccba-460c-b072-a45dd510116b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:31.959316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b7234301-ccba-460c-b072-a45dd510116b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:32.064896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7234301-ccba-460c-b072-a45dd510116b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:32.068188 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7234301-ccba-460c-b072-a45dd510116b service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:32.082354 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1127/2264] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:22:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:22:35.409661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf3b1f-6f33-4899-a4e0-4d3da6d9083a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:35.426816 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf3b1f-6f33-4899-a4e0-4d3da6d9083a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:35.432496 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1f9fa2-c440-49d8-bd4d-fc5ff920f205 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:35.433150 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f9fa2-c440-49d8-bd4d-fc5ff920f205 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:35.433440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f9fa2-c440-49d8-bd4d-fc5ff920f205 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:35.433720 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1f9fa2-c440-49d8-bd4d-fc5ff920f205 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:35.435587 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9c1f9fa2-c440-49d8-bd4d-fc5ff920f205 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:35.446595 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1f9fa2-c440-49d8-bd4d-fc5ff920f205 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:35.448194 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1f9fa2-c440-49d8-bd4d-fc5ff920f205 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:35.457623 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1138/2265] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:22:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:22:35.986911 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7234301-ccba-460c-b072-a45dd510116b service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:35.988467 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b7234301-ccba-460c-b072-a45dd510116b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:36.003078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7234301-ccba-460c-b072-a45dd510116b service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:36.008516 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7994c-8642-4aa6-8049-b2aa49b2769e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:36.009165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7994c-8642-4aa6-8049-b2aa49b2769e service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:36.009446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7994c-8642-4aa6-8049-b2aa49b2769e service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:36.009723 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7994c-8642-4aa6-8049-b2aa49b2769e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:36.011498 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0dc7994c-8642-4aa6-8049-b2aa49b2769e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:36.023334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7994c-8642-4aa6-8049-b2aa49b2769e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:36.025178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7994c-8642-4aa6-8049-b2aa49b2769e service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:36.041066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1128/2266] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:22:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:22:36.682195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f6e24-374b-46be-8878-3c7922bf662c None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:36.683855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f6e24-374b-46be-8878-3c7922bf662c None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:36.684403 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f6e24-374b-46be-8878-3c7922bf662c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:36.713663 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5d4f6e24-374b-46be-8878-3c7922bf662c None None] MFA Rules not processed for user `114d227e9645409e9082cf9b09b05c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:22:36.715211 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5d4f6e24-374b-46be-8878-3c7922bf662c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:36.726534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1139/2267] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:22:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:22:37.151339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7994c-8642-4aa6-8049-b2aa49b2769e service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:37.152943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0dc7994c-8642-4aa6-8049-b2aa49b2769e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:37.168722 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7994c-8642-4aa6-8049-b2aa49b2769e service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:37.174739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7621d-7c98-44ad-ae96-da3dbd7e4284 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:37.176771 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7621d-7c98-44ad-ae96-da3dbd7e4284 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:37.177271 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7621d-7c98-44ad-ae96-da3dbd7e4284 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:37.178756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7621d-7c98-44ad-ae96-da3dbd7e4284 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:37.182613 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-18b7621d-7c98-44ad-ae96-da3dbd7e4284 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:37.202947 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7621d-7c98-44ad-ae96-da3dbd7e4284 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:37.204406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7621d-7c98-44ad-ae96-da3dbd7e4284 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:37.215160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1129/2268] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:22:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:22:37.672229 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f6e24-374b-46be-8878-3c7922bf662c None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:37.678050 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5d4f6e24-374b-46be-8878-3c7922bf662c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:37.767793 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f6e24-374b-46be-8878-3c7922bf662c None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:37.775558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2519aa-3725-4b87-816e-c29afac89f60 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7662e61619d44c92af3ffacbacba3913', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:37.776349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2519aa-3725-4b87-816e-c29afac89f60 service placement] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:37.776737 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2519aa-3725-4b87-816e-c29afac89f60 service placement] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:37.777359 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2519aa-3725-4b87-816e-c29afac89f60 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:37.779536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4e2519aa-3725-4b87-816e-c29afac89f60 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:37.793255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2519aa-3725-4b87-816e-c29afac89f60 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:37.795148 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2519aa-3725-4b87-816e-c29afac89f60 service placement] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:37.806189 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1140/2269] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:22:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:22:42.532927 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a9835-dd42-4f37-88a7-10beb471ab4f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:42.533526 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a9835-dd42-4f37-88a7-10beb471ab4f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:42.533836 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a9835-dd42-4f37-88a7-10beb471ab4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:42.556627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-217a9835-dd42-4f37-88a7-10beb471ab4f None None] MFA Rules not processed for user `98f58f91d17949198e66883d234a9e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:22:42.557903 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-217a9835-dd42-4f37-88a7-10beb471ab4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:42.624811 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1130/2270] 10.222.0.16 () {58 vars in 967 bytes} [Fri Jun 24 04:22:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:22:42.829799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2519aa-3725-4b87-816e-c29afac89f60 service placement] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:42.831408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4e2519aa-3725-4b87-816e-c29afac89f60 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:42.905461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2519aa-3725-4b87-816e-c29afac89f60 service placement] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:42.910690 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146ea94-2652-4262-a7c1-bfadf68a1c95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8e1f1e3c7e42cc9fa6f82aaac55129', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:42.911346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146ea94-2652-4262-a7c1-bfadf68a1c95 service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:42.911641 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146ea94-2652-4262-a7c1-bfadf68a1c95 service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:42.911921 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146ea94-2652-4262-a7c1-bfadf68a1c95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:42.920560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146ea94-2652-4262-a7c1-bfadf68a1c95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:42.922192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146ea94-2652-4262-a7c1-bfadf68a1c95 service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:42.930803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1141/2271] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jun 24 04:22:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 04:22:45.509626 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a9835-dd42-4f37-88a7-10beb471ab4f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:45.524275 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a9835-dd42-4f37-88a7-10beb471ab4f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:45.534076 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf71205-e470-4f3b-b6b3-aa80ce59e697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:45.535011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf71205-e470-4f3b-b6b3-aa80ce59e697 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:45.535729 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf71205-e470-4f3b-b6b3-aa80ce59e697 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:45.536196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf71205-e470-4f3b-b6b3-aa80ce59e697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:45.538606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-aaf71205-e470-4f3b-b6b3-aa80ce59e697 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:45.554447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf71205-e470-4f3b-b6b3-aa80ce59e697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:45.555920 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf71205-e470-4f3b-b6b3-aa80ce59e697 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:45.566132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1131/2272] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:22:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:22:52.927359 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:22:53.422541 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146ea94-2652-4262-a7c1-bfadf68a1c95 service swift] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:53.422541 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4146ea94-2652-4262-a7c1-bfadf68a1c95 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:53.555799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146ea94-2652-4262-a7c1-bfadf68a1c95 service swift] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:53.566001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5295ce-dfae-4124-b5e3-66560a4b7bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:53.568529 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5295ce-dfae-4124-b5e3-66560a4b7bfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:53.568529 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5295ce-dfae-4124-b5e3-66560a4b7bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:53.568529 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5295ce-dfae-4124-b5e3-66560a4b7bfa admin admin] PATH_INFO: `/v3/users/4fb36d1e057543a3abcf6d6c2564d9ae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:53.569401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5295ce-dfae-4124-b5e3-66560a4b7bfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fb36d1e057543a3abcf6d6c2564d9ae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:53.576811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5295ce-dfae-4124-b5e3-66560a4b7bfa admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:53.745697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7c5295ce-dfae-4124-b5e3-66560a4b7bfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fb36d1e057543a3abcf6d6c2564d9ae'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:53.763255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7c5295ce-dfae-4124-b5e3-66560a4b7bfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fb36d1e057543a3abcf6d6c2564d9ae'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:53.765077 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7c5295ce-dfae-4124-b5e3-66560a4b7bfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fb36d1e057543a3abcf6d6c2564d9ae'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:53.766408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7c5295ce-dfae-4124-b5e3-66560a4b7bfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fb36d1e057543a3abcf6d6c2564d9ae'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:53.780071 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7c5295ce-dfae-4124-b5e3-66560a4b7bfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fb36d1e057543a3abcf6d6c2564d9ae'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:53.851837 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1142/2273] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:22:53 2022] DELETE /identity/v3/users/4fb36d1e057543a3abcf6d6c2564d9ae => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:22:54.281109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf71205-e470-4f3b-b6b3-aa80ce59e697 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:54.305703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf71205-e470-4f3b-b6b3-aa80ce59e697 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:54.406026 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89758237-2336-4e1d-911d-9b4e1d49b377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:54.407411 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89758237-2336-4e1d-911d-9b4e1d49b377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:54.407961 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89758237-2336-4e1d-911d-9b4e1d49b377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:54.408528 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89758237-2336-4e1d-911d-9b4e1d49b377 admin admin] PATH_INFO: `/v3/projects/d51a251ca7084b2bb07d808026cccd67` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:54.410904 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89758237-2336-4e1d-911d-9b4e1d49b377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d51a251ca7084b2bb07d808026cccd67)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:54.413940 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89758237-2336-4e1d-911d-9b4e1d49b377 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:54.427797 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:22:54.540880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-89758237-2336-4e1d-911d-9b4e1d49b377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd51a251ca7084b2bb07d808026cccd67', 'request_id': 'req-89758237-2336-4e1d-911d-9b4e1d49b377'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:54.629843 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-89758237-2336-4e1d-911d-9b4e1d49b377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd51a251ca7084b2bb07d808026cccd67', 'request_id': 'req-89758237-2336-4e1d-911d-9b4e1d49b377'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:54.637648 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-89758237-2336-4e1d-911d-9b4e1d49b377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd51a251ca7084b2bb07d808026cccd67', 'request_id': 'req-89758237-2336-4e1d-911d-9b4e1d49b377'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:54.658518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-89758237-2336-4e1d-911d-9b4e1d49b377 admin admin] The token cache is being invalidate because project d51a251ca7084b2bb07d808026cccd67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:22:54.658815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-89758237-2336-4e1d-911d-9b4e1d49b377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:54.659546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-89758237-2336-4e1d-911d-9b4e1d49b377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:54.661744 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1132/2274] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:22:54 2022] DELETE /identity/v3/projects/d51a251ca7084b2bb07d808026cccd67 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:22:54.666557 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5295ce-dfae-4124-b5e3-66560a4b7bfa admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:54.668542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7c5295ce-dfae-4124-b5e3-66560a4b7bfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:54.690813 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5295ce-dfae-4124-b5e3-66560a4b7bfa admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:54.788102 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587e387-f89f-4134-bb59-618859c9528a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:54.789073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587e387-f89f-4134-bb59-618859c9528a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:54.789565 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587e387-f89f-4134-bb59-618859c9528a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:54.789916 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587e387-f89f-4134-bb59-618859c9528a admin admin] PATH_INFO: `/v3/users/3ec36b2f17324fe699593c3aebb6c39c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:54.791305 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587e387-f89f-4134-bb59-618859c9528a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ec36b2f17324fe699593c3aebb6c39c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:54.793392 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587e387-f89f-4134-bb59-618859c9528a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:54.809606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89758237-2336-4e1d-911d-9b4e1d49b377 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:54.814793 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-89758237-2336-4e1d-911d-9b4e1d49b377 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:54.821267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:22:54.913925 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89758237-2336-4e1d-911d-9b4e1d49b377 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:54.929404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61b11a-d57c-4dbc-9c6b-30ac9a7542b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:54.930088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61b11a-d57c-4dbc-9c6b-30ac9a7542b9 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:54.930400 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61b11a-d57c-4dbc-9c6b-30ac9a7542b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:54.930686 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61b11a-d57c-4dbc-9c6b-30ac9a7542b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:54.932560 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2f61b11a-d57c-4dbc-9c6b-30ac9a7542b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:55.002281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61b11a-d57c-4dbc-9c6b-30ac9a7542b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:55.003879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61b11a-d57c-4dbc-9c6b-30ac9a7542b9 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:55.013404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1133/2275] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:22:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:22:55.078645 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0587e387-f89f-4134-bb59-618859c9528a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ec36b2f17324fe699593c3aebb6c39c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:55.090469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0587e387-f89f-4134-bb59-618859c9528a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ec36b2f17324fe699593c3aebb6c39c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:55.091165 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0587e387-f89f-4134-bb59-618859c9528a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ec36b2f17324fe699593c3aebb6c39c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:55.091848 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0587e387-f89f-4134-bb59-618859c9528a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ec36b2f17324fe699593c3aebb6c39c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:55.099933 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0587e387-f89f-4134-bb59-618859c9528a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ec36b2f17324fe699593c3aebb6c39c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:55.196508 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1143/2276] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:22:54 2022] DELETE /identity/v3/users/3ec36b2f17324fe699593c3aebb6c39c => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:22:55.450767 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61b11a-d57c-4dbc-9c6b-30ac9a7542b9 service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:55.452459 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2f61b11a-d57c-4dbc-9c6b-30ac9a7542b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:55.466956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61b11a-d57c-4dbc-9c6b-30ac9a7542b9 service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:55.501247 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:22:55.528950 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa837c16-6a30-44f1-8b62-f765f1b7486f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:55.529745 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa837c16-6a30-44f1-8b62-f765f1b7486f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:55.530772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa837c16-6a30-44f1-8b62-f765f1b7486f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:55.531254 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa837c16-6a30-44f1-8b62-f765f1b7486f admin admin] PATH_INFO: `/v3/projects/d308801b882b46209e83593e0607ee89` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:55.532317 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa837c16-6a30-44f1-8b62-f765f1b7486f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d308801b882b46209e83593e0607ee89)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:55.534278 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa837c16-6a30-44f1-8b62-f765f1b7486f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:55.713303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fa837c16-6a30-44f1-8b62-f765f1b7486f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd308801b882b46209e83593e0607ee89', 'request_id': 'req-fa837c16-6a30-44f1-8b62-f765f1b7486f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:55.820520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fa837c16-6a30-44f1-8b62-f765f1b7486f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd308801b882b46209e83593e0607ee89', 'request_id': 'req-fa837c16-6a30-44f1-8b62-f765f1b7486f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:55.826645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fa837c16-6a30-44f1-8b62-f765f1b7486f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd308801b882b46209e83593e0607ee89', 'request_id': 'req-fa837c16-6a30-44f1-8b62-f765f1b7486f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:55.840175 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fa837c16-6a30-44f1-8b62-f765f1b7486f admin admin] The token cache is being invalidate because project d308801b882b46209e83593e0607ee89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:22:55.840467 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fa837c16-6a30-44f1-8b62-f765f1b7486f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:55.841139 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fa837c16-6a30-44f1-8b62-f765f1b7486f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:22:55.843256 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1134/2277] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:22:55 2022] DELETE /identity/v3/projects/d308801b882b46209e83593e0607ee89 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:22:55.848179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98c3b4-e7eb-4db2-8e30-c703db5b5048 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:55.848627 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98c3b4-e7eb-4db2-8e30-c703db5b5048 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:55.848742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98c3b4-e7eb-4db2-8e30-c703db5b5048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:55.867850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3b98c3b4-e7eb-4db2-8e30-c703db5b5048 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:22:55.869074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3b98c3b4-e7eb-4db2-8e30-c703db5b5048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:55.938154 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1144/2278] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:22:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:22:55.946204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9ee42-9ade-4b12-81a2-25dccda6ea1f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:55.946602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9ee42-9ade-4b12-81a2-25dccda6ea1f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:55.946965 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9ee42-9ade-4b12-81a2-25dccda6ea1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:55.966016 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-93f9ee42-9ade-4b12-81a2-25dccda6ea1f None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:22:55.967280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-93f9ee42-9ade-4b12-81a2-25dccda6ea1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:55.975655 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1135/2279] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:22:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:22:55.979910 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98c3b4-e7eb-4db2-8e30-c703db5b5048 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:55.990725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98c3b4-e7eb-4db2-8e30-c703db5b5048 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:55.995810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7fd96-a8d6-4f04-be22-9702d012e04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:55.996478 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7fd96-a8d6-4f04-be22-9702d012e04f admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:55.996799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7fd96-a8d6-4f04-be22-9702d012e04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:55.997084 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7fd96-a8d6-4f04-be22-9702d012e04f admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:55.997821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7fd96-a8d6-4f04-be22-9702d012e04f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:22:55.998140 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7fd96-a8d6-4f04-be22-9702d012e04f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:56.000612 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7fd96-a8d6-4f04-be22-9702d012e04f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:56.014822 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1145/2280] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:22:56.019188 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9ee42-9ade-4b12-81a2-25dccda6ea1f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:56.035361 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9ee42-9ade-4b12-81a2-25dccda6ea1f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:56.049032 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9e340-2ca3-450a-9002-70f3a90ac07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:56.049696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9e340-2ca3-450a-9002-70f3a90ac07a admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:56.049984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9e340-2ca3-450a-9002-70f3a90ac07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:56.050259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9e340-2ca3-450a-9002-70f3a90ac07a admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:56.051057 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9e340-2ca3-450a-9002-70f3a90ac07a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:56.053298 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b9e340-2ca3-450a-9002-70f3a90ac07a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:56.152609 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1136/2281] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:22:56 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:22:56.161216 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7fd96-a8d6-4f04-be22-9702d012e04f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:56.172300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7fd96-a8d6-4f04-be22-9702d012e04f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:56.236133 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac3235-fa6e-445b-8725-e2b54dbdcacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:56.236829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac3235-fa6e-445b-8725-e2b54dbdcacb admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:56.237119 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac3235-fa6e-445b-8725-e2b54dbdcacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:56.237397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac3235-fa6e-445b-8725-e2b54dbdcacb admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:56.238107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac3235-fa6e-445b-8725-e2b54dbdcacb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:56.240353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac3235-fa6e-445b-8725-e2b54dbdcacb admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:56.352317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1146/2282] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:22:56 2022] POST /identity/v3/users => generated 290 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:22:56.356429 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9e340-2ca3-450a-9002-70f3a90ac07a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:56.367737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9e340-2ca3-450a-9002-70f3a90ac07a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:56.372421 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa9b89-484b-4c5b-8ecd-3d8f98a92642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:56.373461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa9b89-484b-4c5b-8ecd-3d8f98a92642 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:56.373461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa9b89-484b-4c5b-8ecd-3d8f98a92642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:56.374338 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aa9b89-484b-4c5b-8ecd-3d8f98a92642 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:56.375228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa9b89-484b-4c5b-8ecd-3d8f98a92642 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:56.376845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aa9b89-484b-4c5b-8ecd-3d8f98a92642 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:56.392615 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1137/2283] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:22:56.393712 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac3235-fa6e-445b-8725-e2b54dbdcacb admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:56.405485 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac3235-fa6e-445b-8725-e2b54dbdcacb admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:56.410318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d38b33-30a0-4896-bdc2-96546f37df18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:56.412382 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d38b33-30a0-4896-bdc2-96546f37df18 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:56.412382 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d38b33-30a0-4896-bdc2-96546f37df18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:56.412382 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d38b33-30a0-4896-bdc2-96546f37df18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:56.412382 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d38b33-30a0-4896-bdc2-96546f37df18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:56.413888 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d38b33-30a0-4896-bdc2-96546f37df18 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:56.424721 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1147/2284] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:22:56.429164 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa9b89-484b-4c5b-8ecd-3d8f98a92642 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:56.440389 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aa9b89-484b-4c5b-8ecd-3d8f98a92642 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:56.444953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da741f11-fd1a-4d33-bee6-4e267167de2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:56.445853 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da741f11-fd1a-4d33-bee6-4e267167de2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:56.446169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da741f11-fd1a-4d33-bee6-4e267167de2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:56.446482 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da741f11-fd1a-4d33-bee6-4e267167de2c admin admin] PATH_INFO: `/v3/projects/f9c12f0aa786483e8d1ce8ff9042164f/users/d1cee676e3dc47e18e94de7e20b59455/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:56.463632 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da741f11-fd1a-4d33-bee6-4e267167de2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9c12f0aa786483e8d1ce8ff9042164f, user_id=d1cee676e3dc47e18e94de7e20b59455, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:56.466497 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da741f11-fd1a-4d33-bee6-4e267167de2c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:56.584936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1138/2285] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:22:56 2022] PUT /identity/v3/projects/f9c12f0aa786483e8d1ce8ff9042164f/users/d1cee676e3dc47e18e94de7e20b59455/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:22:56.590145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fa75c-3579-40a9-bec6-a709414acbd5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:56.590478 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fa75c-3579-40a9-bec6-a709414acbd5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:56.590763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fa75c-3579-40a9-bec6-a709414acbd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:56.599314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:22:56.611214 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-338fa75c-3579-40a9-bec6-a709414acbd5 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:22:56.612645 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-338fa75c-3579-40a9-bec6-a709414acbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:56.679149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1148/2286] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:22:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:22:56.691017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da741f11-fd1a-4d33-bee6-4e267167de2c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:56.693523 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-da741f11-fd1a-4d33-bee6-4e267167de2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:56.761973 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da741f11-fd1a-4d33-bee6-4e267167de2c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:56.767548 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb7ec4-ed00-4173-aa98-489712d84d31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:56.768549 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb7ec4-ed00-4173-aa98-489712d84d31 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:56.769017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb7ec4-ed00-4173-aa98-489712d84d31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:56.769491 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defb7ec4-ed00-4173-aa98-489712d84d31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:56.780562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb7ec4-ed00-4173-aa98-489712d84d31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:56.782310 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defb7ec4-ed00-4173-aa98-489712d84d31 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:56.789271 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:22:56.794828 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1139/2287] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:22:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:22:57.333602 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338fa75c-3579-40a9-bec6-a709414acbd5 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:22:57.335220 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-338fa75c-3579-40a9-bec6-a709414acbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:57.433829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338fa75c-3579-40a9-bec6-a709414acbd5 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:22:57.440166 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941456e-844d-4e02-9e38-8f0e2b153167 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:22:57.441143 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941456e-844d-4e02-9e38-8f0e2b153167 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:22:57.441681 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941456e-844d-4e02-9e38-8f0e2b153167 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:22:57.442143 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941456e-844d-4e02-9e38-8f0e2b153167 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:22:57.444117 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5941456e-844d-4e02-9e38-8f0e2b153167 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:22:57.462656 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941456e-844d-4e02-9e38-8f0e2b153167 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:22:57.464855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941456e-844d-4e02-9e38-8f0e2b153167 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:22:57.479913 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1149/2288] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:22:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:23:01.067139 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb7ec4-ed00-4173-aa98-489712d84d31 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:01.083090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defb7ec4-ed00-4173-aa98-489712d84d31 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:01.090542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa34c4-d71a-48c3-8aae-30ce04c62691 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:01.091216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa34c4-d71a-48c3-8aae-30ce04c62691 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:01.091500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa34c4-d71a-48c3-8aae-30ce04c62691 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:01.091783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa34c4-d71a-48c3-8aae-30ce04c62691 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:01.094901 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0faa34c4-d71a-48c3-8aae-30ce04c62691 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:01.112567 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa34c4-d71a-48c3-8aae-30ce04c62691 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:01.112567 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa34c4-d71a-48c3-8aae-30ce04c62691 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:01.125213 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1140/2289] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:23:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:23:02.954235 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941456e-844d-4e02-9e38-8f0e2b153167 service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:02.956415 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5941456e-844d-4e02-9e38-8f0e2b153167 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:02.973807 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941456e-844d-4e02-9e38-8f0e2b153167 service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:02.977955 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a06601-06b3-405d-a2e4-27d77bae42ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:02.979246 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a06601-06b3-405d-a2e4-27d77bae42ab service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:02.979246 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a06601-06b3-405d-a2e4-27d77bae42ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:02.979556 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a06601-06b3-405d-a2e4-27d77bae42ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:02.983257 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c7a06601-06b3-405d-a2e4-27d77bae42ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:02.998813 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a06601-06b3-405d-a2e4-27d77bae42ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:03.000838 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a06601-06b3-405d-a2e4-27d77bae42ab service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:03.013759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1150/2290] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:23:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:23:03.339891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa05dd3-62d9-42de-8f86-2de07827f212 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:03.340896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa05dd3-62d9-42de-8f86-2de07827f212 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:03.341338 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa05dd3-62d9-42de-8f86-2de07827f212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:03.376220 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-eaa05dd3-62d9-42de-8f86-2de07827f212 None None] MFA Rules not processed for user `d1cee676e3dc47e18e94de7e20b59455`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:03.377756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-eaa05dd3-62d9-42de-8f86-2de07827f212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:03.468726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1141/2291] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:03.473610 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a06601-06b3-405d-a2e4-27d77bae42ab service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:03.485482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a06601-06b3-405d-a2e4-27d77bae42ab service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:03.490260 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077e0e1-e098-44c4-a94b-2c085e940ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:03.491122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077e0e1-e098-44c4-a94b-2c085e940ceb admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:03.491480 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077e0e1-e098-44c4-a94b-2c085e940ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:03.491817 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077e0e1-e098-44c4-a94b-2c085e940ceb admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:03.492579 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077e0e1-e098-44c4-a94b-2c085e940ceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:03.494758 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077e0e1-e098-44c4-a94b-2c085e940ceb admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:03.594103 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1151/2292] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:23:03 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:23:03.598350 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa05dd3-62d9-42de-8f86-2de07827f212 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:03.610663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa05dd3-62d9-42de-8f86-2de07827f212 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:03.679194 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd05d24-9bec-4f5a-a0cb-c4fb26188ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:03.680163 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd05d24-9bec-4f5a-a0cb-c4fb26188ce8 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:03.680641 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd05d24-9bec-4f5a-a0cb-c4fb26188ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:03.680934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd05d24-9bec-4f5a-a0cb-c4fb26188ce8 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:03.681675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd05d24-9bec-4f5a-a0cb-c4fb26188ce8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:03.683702 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd05d24-9bec-4f5a-a0cb-c4fb26188ce8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:03.790065 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1142/2293] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:23:03 2022] POST /identity/v3/users => generated 295 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:23:03.791327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077e0e1-e098-44c4-a94b-2c085e940ceb admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:03.802001 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077e0e1-e098-44c4-a94b-2c085e940ceb admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:03.807008 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f281743-20e5-46d2-9a3d-74c032a71b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:03.807723 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f281743-20e5-46d2-9a3d-74c032a71b26 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:03.808013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f281743-20e5-46d2-9a3d-74c032a71b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:03.808323 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f281743-20e5-46d2-9a3d-74c032a71b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:03.809096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f281743-20e5-46d2-9a3d-74c032a71b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:03.810762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f281743-20e5-46d2-9a3d-74c032a71b26 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:03.822168 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1152/2294] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:23:03.827148 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd05d24-9bec-4f5a-a0cb-c4fb26188ce8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:03.838214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd05d24-9bec-4f5a-a0cb-c4fb26188ce8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:03.843255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ee6ad-e501-4fd8-b295-c5317ce4b19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:03.845271 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ee6ad-e501-4fd8-b295-c5317ce4b19c admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:03.845487 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ee6ad-e501-4fd8-b295-c5317ce4b19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:03.845776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ee6ad-e501-4fd8-b295-c5317ce4b19c admin admin] PATH_INFO: `/v3/projects/3c7828b5281148b89afe2f97d334a613/users/53baf1b3e4634b25b7f188a9843a4ba1/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:03.863001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ee6ad-e501-4fd8-b295-c5317ce4b19c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c7828b5281148b89afe2f97d334a613, user_id=53baf1b3e4634b25b7f188a9843a4ba1, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:03.865750 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ee6ad-e501-4fd8-b295-c5317ce4b19c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:03.935401 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1143/2295] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:23:03 2022] PUT /identity/v3/projects/3c7828b5281148b89afe2f97d334a613/users/53baf1b3e4634b25b7f188a9843a4ba1/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:23:03.939595 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f281743-20e5-46d2-9a3d-74c032a71b26 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:03.949900 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f281743-20e5-46d2-9a3d-74c032a71b26 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:03.997007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:04.013202 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ed03a-b079-44cc-b929-612ee8b0c37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:04.013883 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ed03a-b079-44cc-b929-612ee8b0c37e admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:04.014317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ed03a-b079-44cc-b929-612ee8b0c37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:04.014507 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ed03a-b079-44cc-b929-612ee8b0c37e admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:04.015346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ed03a-b079-44cc-b929-612ee8b0c37e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:04.016965 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ed03a-b079-44cc-b929-612ee8b0c37e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:04.027537 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1153/2296] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:23:04.032710 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ee6ad-e501-4fd8-b295-c5317ce4b19c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:04.048301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ee6ad-e501-4fd8-b295-c5317ce4b19c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:04.053456 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278be325-5d44-4567-90cf-db47337be61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:04.054535 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278be325-5d44-4567-90cf-db47337be61d admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:04.054825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278be325-5d44-4567-90cf-db47337be61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:04.055359 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278be325-5d44-4567-90cf-db47337be61d admin admin] PATH_INFO: `/v3/projects/3c7828b5281148b89afe2f97d334a613/users/53baf1b3e4634b25b7f188a9843a4ba1/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:04.057494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278be325-5d44-4567-90cf-db47337be61d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c7828b5281148b89afe2f97d334a613, user_id=53baf1b3e4634b25b7f188a9843a4ba1, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:04.060261 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278be325-5d44-4567-90cf-db47337be61d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:04.073095 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1144/2297] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:23:04 2022] PUT /identity/v3/projects/3c7828b5281148b89afe2f97d334a613/users/53baf1b3e4634b25b7f188a9843a4ba1/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:23:08.618963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ed03a-b079-44cc-b929-612ee8b0c37e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:08.620781 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5e1ed03a-b079-44cc-b929-612ee8b0c37e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:08.641720 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ed03a-b079-44cc-b929-612ee8b0c37e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:08.747663 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f99aac-ae9a-413c-a0f3-f56df90363c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:08.749299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f99aac-ae9a-413c-a0f3-f56df90363c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:08.749820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f99aac-ae9a-413c-a0f3-f56df90363c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:08.750495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f99aac-ae9a-413c-a0f3-f56df90363c9 admin admin] PATH_INFO: `/v3/users/b158078d039a4979a9c0cf2be26c4b14` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:08.752137 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f99aac-ae9a-413c-a0f3-f56df90363c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b158078d039a4979a9c0cf2be26c4b14)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:08.755671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f99aac-ae9a-413c-a0f3-f56df90363c9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:09.005091 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-73f99aac-ae9a-413c-a0f3-f56df90363c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b158078d039a4979a9c0cf2be26c4b14'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:09.059570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-73f99aac-ae9a-413c-a0f3-f56df90363c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b158078d039a4979a9c0cf2be26c4b14'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:09.062404 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-73f99aac-ae9a-413c-a0f3-f56df90363c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b158078d039a4979a9c0cf2be26c4b14'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:09.063149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-73f99aac-ae9a-413c-a0f3-f56df90363c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b158078d039a4979a9c0cf2be26c4b14'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:09.078127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-73f99aac-ae9a-413c-a0f3-f56df90363c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b158078d039a4979a9c0cf2be26c4b14'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:09.163996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278be325-5d44-4567-90cf-db47337be61d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:09.165844 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-278be325-5d44-4567-90cf-db47337be61d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:09.203641 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1154/2298] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:23:08 2022] DELETE /identity/v3/users/b158078d039a4979a9c0cf2be26c4b14 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:23:09.214862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:09.251349 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278be325-5d44-4567-90cf-db47337be61d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:09.316475 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d5375-04e4-4b9b-bc74-e693653e4ab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:09.317073 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d5375-04e4-4b9b-bc74-e693653e4ab0 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:09.318430 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d5375-04e4-4b9b-bc74-e693653e4ab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:09.318936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d5375-04e4-4b9b-bc74-e693653e4ab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:09.321621 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c33d5375-04e4-4b9b-bc74-e693653e4ab0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:09.332657 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d5375-04e4-4b9b-bc74-e693653e4ab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:09.334122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d5375-04e4-4b9b-bc74-e693653e4ab0 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:09.344250 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1145/2299] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:23:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:23:09.641755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f99aac-ae9a-413c-a0f3-f56df90363c9 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:09.643267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-73f99aac-ae9a-413c-a0f3-f56df90363c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:09.746885 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f99aac-ae9a-413c-a0f3-f56df90363c9 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:09.752180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68ac8e-cdaf-4b0e-990b-000b5bf3bae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:09.753129 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68ac8e-cdaf-4b0e-990b-000b5bf3bae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:09.753417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68ac8e-cdaf-4b0e-990b-000b5bf3bae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:09.753697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68ac8e-cdaf-4b0e-990b-000b5bf3bae5 admin admin] PATH_INFO: `/v3/projects/96e2b5a124bf48fd82e48ed79aa4eeed` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:09.754973 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68ac8e-cdaf-4b0e-990b-000b5bf3bae5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96e2b5a124bf48fd82e48ed79aa4eeed)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:09.756923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68ac8e-cdaf-4b0e-990b-000b5bf3bae5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:09.758656 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:09.849874 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df320e6-1955-4887-981b-421b17e00461 None None] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:09.850252 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df320e6-1955-4887-981b-421b17e00461 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:09.850887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df320e6-1955-4887-981b-421b17e00461 None None] PATH_INFO: `/` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:09.852351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1146/2300] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jun 24 04:23:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:23:09.861529 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a210d-77d1-48c9-b53d-344223c3afc2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:09.861885 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a210d-77d1-48c9-b53d-344223c3afc2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:09.862486 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a210d-77d1-48c9-b53d-344223c3afc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:09.883072 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9b68ac8e-cdaf-4b0e-990b-000b5bf3bae5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96e2b5a124bf48fd82e48ed79aa4eeed', 'request_id': 'req-9b68ac8e-cdaf-4b0e-990b-000b5bf3bae5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:09.891952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e88a210d-77d1-48c9-b53d-344223c3afc2 None None] MFA Rules not processed for user `80b7b3ec60db4de19bbf7d08f111f470`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:09.893269 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e88a210d-77d1-48c9-b53d-344223c3afc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:09.903038 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9b68ac8e-cdaf-4b0e-990b-000b5bf3bae5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96e2b5a124bf48fd82e48ed79aa4eeed', 'request_id': 'req-9b68ac8e-cdaf-4b0e-990b-000b5bf3bae5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:09.927249 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9b68ac8e-cdaf-4b0e-990b-000b5bf3bae5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96e2b5a124bf48fd82e48ed79aa4eeed', 'request_id': 'req-9b68ac8e-cdaf-4b0e-990b-000b5bf3bae5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:10.010581 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1147/2301] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jun 24 04:23:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:23:10.022689 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a210d-77d1-48c9-b53d-344223c3afc2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:10.024996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e88a210d-77d1-48c9-b53d-344223c3afc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:10.056699 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9b68ac8e-cdaf-4b0e-990b-000b5bf3bae5 admin admin] The token cache is being invalidate because project 96e2b5a124bf48fd82e48ed79aa4eeed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:23:10.056699 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9b68ac8e-cdaf-4b0e-990b-000b5bf3bae5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:10.056699 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-9b68ac8e-cdaf-4b0e-990b-000b5bf3bae5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:10.058791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1155/2302] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:23:09 2022] DELETE /identity/v3/projects/96e2b5a124bf48fd82e48ed79aa4eeed => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:23:10.064661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3461476-e879-47f5-a786-bfd75d510fd5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:10.066924 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3461476-e879-47f5-a786-bfd75d510fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:10.067444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3461476-e879-47f5-a786-bfd75d510fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:10.106854 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e3461476-e879-47f5-a786-bfd75d510fd5 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:10.106854 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e3461476-e879-47f5-a786-bfd75d510fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:10.171283 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a210d-77d1-48c9-b53d-344223c3afc2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:10.180958 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa96dce-2c3a-42e1-bdda-20e0807b2950 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:10.181857 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa96dce-2c3a-42e1-bdda-20e0807b2950 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:10.182315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa96dce-2c3a-42e1-bdda-20e0807b2950 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:10.182763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa96dce-2c3a-42e1-bdda-20e0807b2950 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:10.186711 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dfa96dce-2c3a-42e1-bdda-20e0807b2950 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:10.214726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa96dce-2c3a-42e1-bdda-20e0807b2950 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:10.217153 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa96dce-2c3a-42e1-bdda-20e0807b2950 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:10.225982 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1156/2303] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:10.234484 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5b3d0-1507-4a1a-b56e-948ca7f76504 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:10.238336 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5b3d0-1507-4a1a-b56e-948ca7f76504 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:10.238336 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5b3d0-1507-4a1a-b56e-948ca7f76504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:10.238625 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1148/2304] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:23:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:23:10.261544 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa96dce-2c3a-42e1-bdda-20e0807b2950 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:10.263154 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dfa96dce-2c3a-42e1-bdda-20e0807b2950 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:10.272004 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f2b5b3d0-1507-4a1a-b56e-948ca7f76504 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:10.275878 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f2b5b3d0-1507-4a1a-b56e-948ca7f76504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:10.284486 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1157/2305] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:23:10.288810 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa96dce-2c3a-42e1-bdda-20e0807b2950 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:10.290026 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5b3d0-1507-4a1a-b56e-948ca7f76504 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:10.298850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae259b31-04f7-42c5-9692-67b1af610f45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:10.299716 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae259b31-04f7-42c5-9692-67b1af610f45 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:10.300070 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae259b31-04f7-42c5-9692-67b1af610f45 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:10.300835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5b3d0-1507-4a1a-b56e-948ca7f76504 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:10.300964 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae259b31-04f7-42c5-9692-67b1af610f45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:10.303266 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ae259b31-04f7-42c5-9692-67b1af610f45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:10.306586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229c2f6-3adf-4b0b-a0d6-5596107f5eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:10.307383 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229c2f6-3adf-4b0b-a0d6-5596107f5eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:10.307770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229c2f6-3adf-4b0b-a0d6-5596107f5eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:10.308151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229c2f6-3adf-4b0b-a0d6-5596107f5eb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:10.309053 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1229c2f6-3adf-4b0b-a0d6-5596107f5eb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:23:10.309467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1229c2f6-3adf-4b0b-a0d6-5596107f5eb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:10.312772 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1229c2f6-3adf-4b0b-a0d6-5596107f5eb0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:10.320617 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae259b31-04f7-42c5-9692-67b1af610f45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:10.322952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae259b31-04f7-42c5-9692-67b1af610f45 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:10.327396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1158/2306] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:23:10.332553 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229c2f6-3adf-4b0b-a0d6-5596107f5eb0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:10.347515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229c2f6-3adf-4b0b-a0d6-5596107f5eb0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:10.349447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1149/2307] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:23:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:23:10.364907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a71d1-3d09-4284-8a9e-a612cf3dd91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:10.366062 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a71d1-3d09-4284-8a9e-a612cf3dd91e admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:10.366488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a71d1-3d09-4284-8a9e-a612cf3dd91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:10.367124 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a71d1-3d09-4284-8a9e-a612cf3dd91e admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:10.368159 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a71d1-3d09-4284-8a9e-a612cf3dd91e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:10.370235 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a71d1-3d09-4284-8a9e-a612cf3dd91e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:10.451678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1159/2308] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:23:10 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:23:10.455836 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae259b31-04f7-42c5-9692-67b1af610f45 service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:10.468593 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae259b31-04f7-42c5-9692-67b1af610f45 service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:10.474406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:10.537034 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47cdcf-409b-47e1-a83f-e24566189868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:10.538868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47cdcf-409b-47e1-a83f-e24566189868 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:10.538868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47cdcf-409b-47e1-a83f-e24566189868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:10.538868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47cdcf-409b-47e1-a83f-e24566189868 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:10.539134 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47cdcf-409b-47e1-a83f-e24566189868 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:10.541985 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47cdcf-409b-47e1-a83f-e24566189868 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:10.652711 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1150/2309] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:23:10 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:23:10.653639 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a71d1-3d09-4284-8a9e-a612cf3dd91e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:10.667027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a71d1-3d09-4284-8a9e-a612cf3dd91e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:10.675374 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188d918-ee9e-40a6-9353-a3cc729e1d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:10.676178 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188d918-ee9e-40a6-9353-a3cc729e1d6b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:10.676529 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188d918-ee9e-40a6-9353-a3cc729e1d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:10.676933 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a188d918-ee9e-40a6-9353-a3cc729e1d6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:10.677791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188d918-ee9e-40a6-9353-a3cc729e1d6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:10.679633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a188d918-ee9e-40a6-9353-a3cc729e1d6b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:10.699926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1160/2310] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:23:10.702476 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47cdcf-409b-47e1-a83f-e24566189868 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:10.713364 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47cdcf-409b-47e1-a83f-e24566189868 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:10.720804 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45f0d4-c548-4a28-98f7-0d4264375012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:10.721651 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45f0d4-c548-4a28-98f7-0d4264375012 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:10.722073 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45f0d4-c548-4a28-98f7-0d4264375012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:10.722387 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45f0d4-c548-4a28-98f7-0d4264375012 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:10.723717 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45f0d4-c548-4a28-98f7-0d4264375012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:10.728323 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45f0d4-c548-4a28-98f7-0d4264375012 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:10.742416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1151/2311] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:23:10.746530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188d918-ee9e-40a6-9353-a3cc729e1d6b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:10.758856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a188d918-ee9e-40a6-9353-a3cc729e1d6b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:10.764516 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be34cf6-683b-4769-97f3-c9d5d1361d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:10.765485 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be34cf6-683b-4769-97f3-c9d5d1361d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:10.765765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be34cf6-683b-4769-97f3-c9d5d1361d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:10.766040 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be34cf6-683b-4769-97f3-c9d5d1361d16 admin admin] PATH_INFO: `/v3/projects/6031a2f5e18544b68d3caec584330e42/users/2366a2600d2e4821a26c633ddad878e9/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:10.784977 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be34cf6-683b-4769-97f3-c9d5d1361d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6031a2f5e18544b68d3caec584330e42, user_id=2366a2600d2e4821a26c633ddad878e9, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:10.787752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be34cf6-683b-4769-97f3-c9d5d1361d16 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:10.928517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1161/2312] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:23:10 2022] PUT /identity/v3/projects/6031a2f5e18544b68d3caec584330e42/users/2366a2600d2e4821a26c633ddad878e9/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:23:10.939982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cc018-d40b-4f56-a48d-179b8b93c569 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:10.940283 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cc018-d40b-4f56-a48d-179b8b93c569 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:10.940568 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203cc018-d40b-4f56-a48d-179b8b93c569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:11.009114 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-203cc018-d40b-4f56-a48d-179b8b93c569 None None] MFA Rules not processed for user `2366a2600d2e4821a26c633ddad878e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:11.010556 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-203cc018-d40b-4f56-a48d-179b8b93c569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:11.134098 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1152/2313] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:11.148350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be34cf6-683b-4769-97f3-c9d5d1361d16 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:11.179320 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:11.315024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be34cf6-683b-4769-97f3-c9d5d1361d16 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:11.340501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dadb9-601d-48fe-8bfc-23530aacd2f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:11.342749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dadb9-601d-48fe-8bfc-23530aacd2f7 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:11.343339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dadb9-601d-48fe-8bfc-23530aacd2f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:11.343819 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dadb9-601d-48fe-8bfc-23530aacd2f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:11.353936 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0dadb9-601d-48fe-8bfc-23530aacd2f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:11.356421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0dadb9-601d-48fe-8bfc-23530aacd2f7 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:11.390395 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1162/2314] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:23:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:23:12.027342 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36034a-5f21-4276-b890-62866d99d635 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:12.027886 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36034a-5f21-4276-b890-62866d99d635 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:12.027886 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36034a-5f21-4276-b890-62866d99d635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:12.062293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-7d36034a-5f21-4276-b890-62866d99d635 None None] MFA Rules not processed for user `53baf1b3e4634b25b7f188a9843a4ba1`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:12.062293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7d36034a-5f21-4276-b890-62866d99d635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:12.203758 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1153/2315] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:12.218223 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dadb9-601d-48fe-8bfc-23530aacd2f7 service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:12.222605 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8c0dadb9-601d-48fe-8bfc-23530aacd2f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:12.242974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dadb9-601d-48fe-8bfc-23530aacd2f7 service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:12.250141 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89631228-42a4-4187-b9b3-2e05e84d9e8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:12.250840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89631228-42a4-4187-b9b3-2e05e84d9e8e service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:12.251387 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89631228-42a4-4187-b9b3-2e05e84d9e8e service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:12.251865 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89631228-42a4-4187-b9b3-2e05e84d9e8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:12.254573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-89631228-42a4-4187-b9b3-2e05e84d9e8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:12.361840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89631228-42a4-4187-b9b3-2e05e84d9e8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:12.365089 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89631228-42a4-4187-b9b3-2e05e84d9e8e service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:12.378785 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1163/2316] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:23:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:23:13.349055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21726b54-4128-4c31-a498-983902131824 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:13.349543 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21726b54-4128-4c31-a498-983902131824 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:13.349727 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21726b54-4128-4c31-a498-983902131824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:13.356180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:13.368662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-21726b54-4128-4c31-a498-983902131824 None None] MFA Rules not processed for user `98f58f91d17949198e66883d234a9e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:13.369877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-21726b54-4128-4c31-a498-983902131824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:13.427009 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1154/2317] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jun 24 04:23:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:13.439086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89631228-42a4-4187-b9b3-2e05e84d9e8e service nova] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:13.440496 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-89631228-42a4-4187-b9b3-2e05e84d9e8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:13.515277 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89631228-42a4-4187-b9b3-2e05e84d9e8e service nova] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:13.520610 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9d6a8-8c59-4906-b0db-711d4c6841ff service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8e1f1e3c7e42cc9fa6f82aaac55129', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:13.521303 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9d6a8-8c59-4906-b0db-711d4c6841ff service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:13.521560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9d6a8-8c59-4906-b0db-711d4c6841ff service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:13.522017 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9d6a8-8c59-4906-b0db-711d4c6841ff service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:13.532186 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9d6a8-8c59-4906-b0db-711d4c6841ff service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:13.534059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9d6a8-8c59-4906-b0db-711d4c6841ff service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:13.543758 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1164/2318] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jun 24 04:23:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 04:23:21.558727 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21726b54-4128-4c31-a498-983902131824 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:21.628228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21726b54-4128-4c31-a498-983902131824 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:21.633554 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc7466-acc3-4d67-bf08-c17349f21cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:21.634463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc7466-acc3-4d67-bf08-c17349f21cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:21.634756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc7466-acc3-4d67-bf08-c17349f21cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:21.635052 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc7466-acc3-4d67-bf08-c17349f21cd3 admin admin] PATH_INFO: `/v3/users/2366a2600d2e4821a26c633ddad878e9` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:21.637867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc7466-acc3-4d67-bf08-c17349f21cd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2366a2600d2e4821a26c633ddad878e9)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:21.638213 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc7466-acc3-4d67-bf08-c17349f21cd3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:21.815125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-08bc7466-acc3-4d67-bf08-c17349f21cd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2366a2600d2e4821a26c633ddad878e9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:21.833643 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-08bc7466-acc3-4d67-bf08-c17349f21cd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2366a2600d2e4821a26c633ddad878e9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:21.926719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-08bc7466-acc3-4d67-bf08-c17349f21cd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2366a2600d2e4821a26c633ddad878e9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:21.927381 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-08bc7466-acc3-4d67-bf08-c17349f21cd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2366a2600d2e4821a26c633ddad878e9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:21.930845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-08bc7466-acc3-4d67-bf08-c17349f21cd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2366a2600d2e4821a26c633ddad878e9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:21.950694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1155/2319] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:23:21 2022] DELETE /identity/v3/users/2366a2600d2e4821a26c633ddad878e9 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:23:21.952420 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506be5d1-de3d-450a-94f9-34468fc26571 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:21.952939 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506be5d1-de3d-450a-94f9-34468fc26571 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:21.953027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506be5d1-de3d-450a-94f9-34468fc26571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:21.978224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-506be5d1-de3d-450a-94f9-34468fc26571 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:21.979668 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-506be5d1-de3d-450a-94f9-34468fc26571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:22.009588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:22.068880 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1165/2320] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:22.082132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc7466-acc3-4d67-bf08-c17349f21cd3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:22.083843 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-08bc7466-acc3-4d67-bf08-c17349f21cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:22.124651 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:22.174716 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc7466-acc3-4d67-bf08-c17349f21cd3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:22.180621 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f932d7-7fbb-4950-98b1-5216921fd3ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:22.180621 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f932d7-7fbb-4950-98b1-5216921fd3ce service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:22.180621 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f932d7-7fbb-4950-98b1-5216921fd3ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:22.180621 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f932d7-7fbb-4950-98b1-5216921fd3ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:22.194137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f932d7-7fbb-4950-98b1-5216921fd3ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:22.214339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f932d7-7fbb-4950-98b1-5216921fd3ce service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:22.214339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1156/2321] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:23:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:23:22.926711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506be5d1-de3d-450a-94f9-34468fc26571 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:22.928197 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-506be5d1-de3d-450a-94f9-34468fc26571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:22.971190 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506be5d1-de3d-450a-94f9-34468fc26571 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:22.979310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b3dfb-886a-48db-9ab6-79e13d4bb972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:22.985115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b3dfb-886a-48db-9ab6-79e13d4bb972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:22.985115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b3dfb-886a-48db-9ab6-79e13d4bb972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:22.985115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b3dfb-886a-48db-9ab6-79e13d4bb972 admin admin] PATH_INFO: `/v3/projects/6031a2f5e18544b68d3caec584330e42` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:22.985115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b3dfb-886a-48db-9ab6-79e13d4bb972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6031a2f5e18544b68d3caec584330e42)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:22.985115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b3dfb-886a-48db-9ab6-79e13d4bb972 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:23.126163 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-128b3dfb-886a-48db-9ab6-79e13d4bb972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6031a2f5e18544b68d3caec584330e42', 'request_id': 'req-128b3dfb-886a-48db-9ab6-79e13d4bb972'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:23.132981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-128b3dfb-886a-48db-9ab6-79e13d4bb972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6031a2f5e18544b68d3caec584330e42', 'request_id': 'req-128b3dfb-886a-48db-9ab6-79e13d4bb972'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:23.145812 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-128b3dfb-886a-48db-9ab6-79e13d4bb972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6031a2f5e18544b68d3caec584330e42', 'request_id': 'req-128b3dfb-886a-48db-9ab6-79e13d4bb972'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:23.215964 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-128b3dfb-886a-48db-9ab6-79e13d4bb972 admin admin] The token cache is being invalidate because project 6031a2f5e18544b68d3caec584330e42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:23:23.216181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-128b3dfb-886a-48db-9ab6-79e13d4bb972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:23.218505 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-128b3dfb-886a-48db-9ab6-79e13d4bb972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:23.220363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1166/2322] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:23:22 2022] DELETE /identity/v3/projects/6031a2f5e18544b68d3caec584330e42 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:23:23.225745 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ae39a-20ba-4766-b208-a8c83c77acf5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:23.225745 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ae39a-20ba-4766-b208-a8c83c77acf5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:23.226316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ae39a-20ba-4766-b208-a8c83c77acf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:23.249691 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-318ae39a-20ba-4766-b208-a8c83c77acf5 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:23.250954 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-318ae39a-20ba-4766-b208-a8c83c77acf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:23.345023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1157/2323] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:23.361514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c0dc6-221c-42f7-883f-dba4ec023872 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:23.361514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c0dc6-221c-42f7-883f-dba4ec023872 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:23.361514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c0dc6-221c-42f7-883f-dba4ec023872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:23.389450 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-370c0dc6-221c-42f7-883f-dba4ec023872 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:23.392015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-370c0dc6-221c-42f7-883f-dba4ec023872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:23.412798 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1167/2324] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:23.414167 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ae39a-20ba-4766-b208-a8c83c77acf5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:23.441635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ae39a-20ba-4766-b208-a8c83c77acf5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:23.450026 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88992797-d457-469f-9a57-29d6818f1c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:23.451494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88992797-d457-469f-9a57-29d6818f1c3c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:23.451782 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88992797-d457-469f-9a57-29d6818f1c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:23.452218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88992797-d457-469f-9a57-29d6818f1c3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:23.453051 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88992797-d457-469f-9a57-29d6818f1c3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:23:23.453397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88992797-d457-469f-9a57-29d6818f1c3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:23.455376 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88992797-d457-469f-9a57-29d6818f1c3c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:23.485915 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c0dc6-221c-42f7-883f-dba4ec023872 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:23.495650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1158/2325] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:23:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:23:23.499319 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c0dc6-221c-42f7-883f-dba4ec023872 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:23.505284 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8adb5-04f9-4aad-b9fa-317ad00bc3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:23.506029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8adb5-04f9-4aad-b9fa-317ad00bc3c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:23.506441 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8adb5-04f9-4aad-b9fa-317ad00bc3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:23.507312 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8adb5-04f9-4aad-b9fa-317ad00bc3c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:23.509308 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d8adb5-04f9-4aad-b9fa-317ad00bc3c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:23.518098 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d8adb5-04f9-4aad-b9fa-317ad00bc3c1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:23.621414 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88992797-d457-469f-9a57-29d6818f1c3c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:23.633051 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1168/2326] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:23:23 2022] POST /identity/v3/projects => generated 387 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:23:23.637749 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88992797-d457-469f-9a57-29d6818f1c3c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:23.766485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764e402-5900-4762-ad0a-061375e6fe3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:23.767309 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764e402-5900-4762-ad0a-061375e6fe3f admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:23.768049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764e402-5900-4762-ad0a-061375e6fe3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:23.768551 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a764e402-5900-4762-ad0a-061375e6fe3f admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:23.769627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764e402-5900-4762-ad0a-061375e6fe3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:23.771884 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a764e402-5900-4762-ad0a-061375e6fe3f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:23.893442 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1159/2327] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:23:23 2022] POST /identity/v3/users => generated 294 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:23:23.897519 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8adb5-04f9-4aad-b9fa-317ad00bc3c1 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:23.910499 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8adb5-04f9-4aad-b9fa-317ad00bc3c1 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:23.916466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b8f4b-52a1-4da6-b54a-7be43e60737b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:23.917143 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b8f4b-52a1-4da6-b54a-7be43e60737b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:23.917425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b8f4b-52a1-4da6-b54a-7be43e60737b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:23.917710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b8f4b-52a1-4da6-b54a-7be43e60737b admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:23.918476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b8f4b-52a1-4da6-b54a-7be43e60737b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:23.920268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b8f4b-52a1-4da6-b54a-7be43e60737b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:23.938539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1169/2328] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:23:23.943392 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764e402-5900-4762-ad0a-061375e6fe3f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:23.960943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a764e402-5900-4762-ad0a-061375e6fe3f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:23.971115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f7a8c-1bc2-4391-adaa-3b8b4c04af92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:23.971909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f7a8c-1bc2-4391-adaa-3b8b4c04af92 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:23.972200 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f7a8c-1bc2-4391-adaa-3b8b4c04af92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:23.972773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f7a8c-1bc2-4391-adaa-3b8b4c04af92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:23.973735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f7a8c-1bc2-4391-adaa-3b8b4c04af92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:23.975485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f7a8c-1bc2-4391-adaa-3b8b4c04af92 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:23.991658 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1160/2329] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:23:23.994011 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b8f4b-52a1-4da6-b54a-7be43e60737b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:24.006893 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b8f4b-52a1-4da6-b54a-7be43e60737b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:24.012340 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b2d7f-ad9d-4004-965e-6beaaa8ec0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:24.014026 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b2d7f-ad9d-4004-965e-6beaaa8ec0cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:24.015132 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b2d7f-ad9d-4004-965e-6beaaa8ec0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:24.015517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b2d7f-ad9d-4004-965e-6beaaa8ec0cd admin admin] PATH_INFO: `/v3/projects/07777eef03ae404b939885b28b2b41ab/users/65c27ff7fba34b6f812dd2935e8ecf6d/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:24.039389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b2d7f-ad9d-4004-965e-6beaaa8ec0cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=07777eef03ae404b939885b28b2b41ab, user_id=65c27ff7fba34b6f812dd2935e8ecf6d, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:24.042539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b2d7f-ad9d-4004-965e-6beaaa8ec0cd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:24.130895 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1170/2330] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:23:23 2022] PUT /identity/v3/projects/07777eef03ae404b939885b28b2b41ab/users/65c27ff7fba34b6f812dd2935e8ecf6d/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:23:24.132806 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add57b79-5787-4e0c-9af6-e28bc00e8e55 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:24.133181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add57b79-5787-4e0c-9af6-e28bc00e8e55 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:24.133571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add57b79-5787-4e0c-9af6-e28bc00e8e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:24.169541 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-add57b79-5787-4e0c-9af6-e28bc00e8e55 None None] MFA Rules not processed for user `65c27ff7fba34b6f812dd2935e8ecf6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:24.169541 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-add57b79-5787-4e0c-9af6-e28bc00e8e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:24.201788 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:24.254834 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1161/2331] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:24.259035 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b2d7f-ad9d-4004-965e-6beaaa8ec0cd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:24.322444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:24.339519 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b2d7f-ad9d-4004-965e-6beaaa8ec0cd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:24.345769 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85b96b-e4c2-4599-bc11-196b173bf8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:24.346676 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85b96b-e4c2-4599-bc11-196b173bf8c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:24.347078 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85b96b-e4c2-4599-bc11-196b173bf8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:24.347493 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85b96b-e4c2-4599-bc11-196b173bf8c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:24.348352 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85b96b-e4c2-4599-bc11-196b173bf8c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:24.350595 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85b96b-e4c2-4599-bc11-196b173bf8c0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:24.827055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1171/2332] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:23:24 2022] POST /identity/v3/projects => generated 385 bytes in 569 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:23:24.833789 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add57b79-5787-4e0c-9af6-e28bc00e8e55 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:24.846807 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add57b79-5787-4e0c-9af6-e28bc00e8e55 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:24.908482 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc605109-27af-44e4-992b-0ff08a9edc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:24.912059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc605109-27af-44e4-992b-0ff08a9edc6c admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:24.912356 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc605109-27af-44e4-992b-0ff08a9edc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:24.914538 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc605109-27af-44e4-992b-0ff08a9edc6c admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:24.914538 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc605109-27af-44e4-992b-0ff08a9edc6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:24.915363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc605109-27af-44e4-992b-0ff08a9edc6c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:25.052851 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1162/2333] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:23:24 2022] POST /identity/v3/users => generated 299 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:23:25.053583 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85b96b-e4c2-4599-bc11-196b173bf8c0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:25.065399 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85b96b-e4c2-4599-bc11-196b173bf8c0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:25.075754 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea011645-d1b7-467f-acf1-546985d7c47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:25.075754 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea011645-d1b7-467f-acf1-546985d7c47d admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:25.075754 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea011645-d1b7-467f-acf1-546985d7c47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:25.075754 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea011645-d1b7-467f-acf1-546985d7c47d admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:25.076740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea011645-d1b7-467f-acf1-546985d7c47d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:25.076740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea011645-d1b7-467f-acf1-546985d7c47d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:25.091179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1172/2334] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:23:25.094932 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc605109-27af-44e4-992b-0ff08a9edc6c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:25.109989 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc605109-27af-44e4-992b-0ff08a9edc6c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:25.118235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0e768-e68f-4616-9831-20ae08fb05a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:25.118235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0e768-e68f-4616-9831-20ae08fb05a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:25.118235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0e768-e68f-4616-9831-20ae08fb05a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:25.119143 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0e768-e68f-4616-9831-20ae08fb05a3 admin admin] PATH_INFO: `/v3/projects/c5bf72451ce9448285e7e5d2676200c1/users/d37e4f67a5c44ab49dacead7a94cd591/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:25.151438 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d0e768-e68f-4616-9831-20ae08fb05a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5bf72451ce9448285e7e5d2676200c1, user_id=d37e4f67a5c44ab49dacead7a94cd591, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:25.155484 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d0e768-e68f-4616-9831-20ae08fb05a3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:25.229501 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1163/2335] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:23:25 2022] PUT /identity/v3/projects/c5bf72451ce9448285e7e5d2676200c1/users/d37e4f67a5c44ab49dacead7a94cd591/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:23:25.233776 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea011645-d1b7-467f-acf1-546985d7c47d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:25.247088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea011645-d1b7-467f-acf1-546985d7c47d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:25.318829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5948f19-2ca8-4783-8b92-1fb2c1f6593f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:25.319707 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5948f19-2ca8-4783-8b92-1fb2c1f6593f admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:25.321863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5948f19-2ca8-4783-8b92-1fb2c1f6593f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:25.321863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5948f19-2ca8-4783-8b92-1fb2c1f6593f admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:25.321863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5948f19-2ca8-4783-8b92-1fb2c1f6593f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:25.322819 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5948f19-2ca8-4783-8b92-1fb2c1f6593f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:25.338454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1173/2336] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:23:25.339952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0e768-e68f-4616-9831-20ae08fb05a3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:25.354245 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0e768-e68f-4616-9831-20ae08fb05a3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:25.361018 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037fc1a-09c9-459c-862e-e25560351218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:25.361934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037fc1a-09c9-459c-862e-e25560351218 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:25.362216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037fc1a-09c9-459c-862e-e25560351218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:25.362533 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037fc1a-09c9-459c-862e-e25560351218 admin admin] PATH_INFO: `/v3/projects/c5bf72451ce9448285e7e5d2676200c1/users/d37e4f67a5c44ab49dacead7a94cd591/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:25.366363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037fc1a-09c9-459c-862e-e25560351218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5bf72451ce9448285e7e5d2676200c1, user_id=d37e4f67a5c44ab49dacead7a94cd591, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:25.369209 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037fc1a-09c9-459c-862e-e25560351218 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:25.384965 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1164/2337] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:23:25 2022] PUT /identity/v3/projects/c5bf72451ce9448285e7e5d2676200c1/users/d37e4f67a5c44ab49dacead7a94cd591/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:23:25.396976 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357df54e-4f21-4394-8619-029e3cdd7310 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:25.397275 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357df54e-4f21-4394-8619-029e3cdd7310 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:25.397559 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357df54e-4f21-4394-8619-029e3cdd7310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:25.432650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-357df54e-4f21-4394-8619-029e3cdd7310 None None] MFA Rules not processed for user `d37e4f67a5c44ab49dacead7a94cd591`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:25.433916 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-357df54e-4f21-4394-8619-029e3cdd7310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:25.540901 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1174/2338] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:25.559125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037fc1a-09c9-459c-862e-e25560351218 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:25.563962 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f037fc1a-09c9-459c-862e-e25560351218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:25.693174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037fc1a-09c9-459c-862e-e25560351218 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:25.714015 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0caf21a-a7f3-444e-80ae-73e3447c4a0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:25.714015 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0caf21a-a7f3-444e-80ae-73e3447c4a0f service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:25.714015 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0caf21a-a7f3-444e-80ae-73e3447c4a0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:25.714015 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0caf21a-a7f3-444e-80ae-73e3447c4a0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:25.742799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0caf21a-a7f3-444e-80ae-73e3447c4a0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:25.744977 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0caf21a-a7f3-444e-80ae-73e3447c4a0f service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:25.769754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1165/2339] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:23:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:23:25.892540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357df54e-4f21-4394-8619-029e3cdd7310 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:25.892540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:25.892540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357df54e-4f21-4394-8619-029e3cdd7310 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:25.892540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22378c2-e6e4-46d9-a6f1-74e1cec99031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:25.893578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22378c2-e6e4-46d9-a6f1-74e1cec99031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:25.893578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22378c2-e6e4-46d9-a6f1-74e1cec99031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:25.893578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22378c2-e6e4-46d9-a6f1-74e1cec99031 admin admin] PATH_INFO: `/v3/users/65c27ff7fba34b6f812dd2935e8ecf6d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:25.894649 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22378c2-e6e4-46d9-a6f1-74e1cec99031 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65c27ff7fba34b6f812dd2935e8ecf6d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:25.897175 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22378c2-e6e4-46d9-a6f1-74e1cec99031 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:26.244846 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f22378c2-e6e4-46d9-a6f1-74e1cec99031 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65c27ff7fba34b6f812dd2935e8ecf6d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:26.263428 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f22378c2-e6e4-46d9-a6f1-74e1cec99031 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65c27ff7fba34b6f812dd2935e8ecf6d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:26.264519 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f22378c2-e6e4-46d9-a6f1-74e1cec99031 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65c27ff7fba34b6f812dd2935e8ecf6d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:26.265160 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f22378c2-e6e4-46d9-a6f1-74e1cec99031 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65c27ff7fba34b6f812dd2935e8ecf6d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:26.281569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f22378c2-e6e4-46d9-a6f1-74e1cec99031 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65c27ff7fba34b6f812dd2935e8ecf6d'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:26.363089 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1175/2340] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:23:25 2022] DELETE /identity/v3/users/65c27ff7fba34b6f812dd2935e8ecf6d => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:23:26.368107 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dcc8c-c217-4eb3-8908-d107f65795c6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:26.368956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dcc8c-c217-4eb3-8908-d107f65795c6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:26.372332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dcc8c-c217-4eb3-8908-d107f65795c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:26.390263 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a57dcc8c-c217-4eb3-8908-d107f65795c6 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:26.391543 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a57dcc8c-c217-4eb3-8908-d107f65795c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:26.406125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:26.461424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1166/2341] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:23:26.471122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22378c2-e6e4-46d9-a6f1-74e1cec99031 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:26.472762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f22378c2-e6e4-46d9-a6f1-74e1cec99031 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:26.556373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22378c2-e6e4-46d9-a6f1-74e1cec99031 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:26.560721 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f50256-00af-4488-9d61-968700e706df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:26.561400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f50256-00af-4488-9d61-968700e706df service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:26.561684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f50256-00af-4488-9d61-968700e706df service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:26.561987 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f50256-00af-4488-9d61-968700e706df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:26.571701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f50256-00af-4488-9d61-968700e706df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:26.573170 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f50256-00af-4488-9d61-968700e706df service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:26.584388 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1176/2342] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:23:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:23:27.068763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57dcc8c-c217-4eb3-8908-d107f65795c6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:27.070637 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a57dcc8c-c217-4eb3-8908-d107f65795c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:27.085023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57dcc8c-c217-4eb3-8908-d107f65795c6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:27.095480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2b18e-2c17-4b8b-8aca-e19782b7bd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:27.096519 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2b18e-2c17-4b8b-8aca-e19782b7bd97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:27.096976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2b18e-2c17-4b8b-8aca-e19782b7bd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:27.097513 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b2b18e-2c17-4b8b-8aca-e19782b7bd97 admin admin] PATH_INFO: `/v3/projects/07777eef03ae404b939885b28b2b41ab` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:27.098877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2b18e-2c17-4b8b-8aca-e19782b7bd97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07777eef03ae404b939885b28b2b41ab)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:27.100919 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b2b18e-2c17-4b8b-8aca-e19782b7bd97 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:27.209136 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c8b2b18e-2c17-4b8b-8aca-e19782b7bd97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07777eef03ae404b939885b28b2b41ab', 'request_id': 'req-c8b2b18e-2c17-4b8b-8aca-e19782b7bd97'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:27.276891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c8b2b18e-2c17-4b8b-8aca-e19782b7bd97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07777eef03ae404b939885b28b2b41ab', 'request_id': 'req-c8b2b18e-2c17-4b8b-8aca-e19782b7bd97'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:27.284520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c8b2b18e-2c17-4b8b-8aca-e19782b7bd97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07777eef03ae404b939885b28b2b41ab', 'request_id': 'req-c8b2b18e-2c17-4b8b-8aca-e19782b7bd97'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:27.299505 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c8b2b18e-2c17-4b8b-8aca-e19782b7bd97 admin admin] The token cache is being invalidate because project 07777eef03ae404b939885b28b2b41ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:23:27.299825 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c8b2b18e-2c17-4b8b-8aca-e19782b7bd97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:27.300502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c8b2b18e-2c17-4b8b-8aca-e19782b7bd97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:27.302120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1167/2343] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:23:27 2022] DELETE /identity/v3/projects/07777eef03ae404b939885b28b2b41ab => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:23:27.306400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f50256-00af-4488-9d61-968700e706df service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:27.316974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f50256-00af-4488-9d61-968700e706df service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:27.393875 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49022b0e-fa41-4c09-b563-46a01d19cbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:27.395501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49022b0e-fa41-4c09-b563-46a01d19cbd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:27.396028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49022b0e-fa41-4c09-b563-46a01d19cbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:27.396465 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49022b0e-fa41-4c09-b563-46a01d19cbd1 admin admin] PATH_INFO: `/v3/users/d37e4f67a5c44ab49dacead7a94cd591` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:27.398154 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49022b0e-fa41-4c09-b563-46a01d19cbd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d37e4f67a5c44ab49dacead7a94cd591)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:27.401164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49022b0e-fa41-4c09-b563-46a01d19cbd1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:27.410013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:27.668873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-49022b0e-fa41-4c09-b563-46a01d19cbd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd37e4f67a5c44ab49dacead7a94cd591'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:27.683070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-49022b0e-fa41-4c09-b563-46a01d19cbd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd37e4f67a5c44ab49dacead7a94cd591'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:27.683855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-49022b0e-fa41-4c09-b563-46a01d19cbd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd37e4f67a5c44ab49dacead7a94cd591'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:27.684498 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-49022b0e-fa41-4c09-b563-46a01d19cbd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd37e4f67a5c44ab49dacead7a94cd591'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:27.692903 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-49022b0e-fa41-4c09-b563-46a01d19cbd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd37e4f67a5c44ab49dacead7a94cd591'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:27.795434 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1177/2344] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:23:27 2022] DELETE /identity/v3/users/d37e4f67a5c44ab49dacead7a94cd591 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:23:28.209444 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2b18e-2c17-4b8b-8aca-e19782b7bd97 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:28.211317 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c8b2b18e-2c17-4b8b-8aca-e19782b7bd97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:28.225582 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b2b18e-2c17-4b8b-8aca-e19782b7bd97 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:28.290262 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2a823-c791-47c6-b9b5-867173385b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:28.291081 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2a823-c791-47c6-b9b5-867173385b6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:28.295652 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2a823-c791-47c6-b9b5-867173385b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:28.295652 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2a823-c791-47c6-b9b5-867173385b6b admin admin] PATH_INFO: `/v3/projects/c5bf72451ce9448285e7e5d2676200c1` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:28.295652 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2a823-c791-47c6-b9b5-867173385b6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5bf72451ce9448285e7e5d2676200c1)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:28.295652 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2a823-c791-47c6-b9b5-867173385b6b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:28.435858 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:28.438793 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a4c2a823-c791-47c6-b9b5-867173385b6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5bf72451ce9448285e7e5d2676200c1', 'request_id': 'req-a4c2a823-c791-47c6-b9b5-867173385b6b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:28.534140 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a4c2a823-c791-47c6-b9b5-867173385b6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5bf72451ce9448285e7e5d2676200c1', 'request_id': 'req-a4c2a823-c791-47c6-b9b5-867173385b6b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:28.540697 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a4c2a823-c791-47c6-b9b5-867173385b6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5bf72451ce9448285e7e5d2676200c1', 'request_id': 'req-a4c2a823-c791-47c6-b9b5-867173385b6b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:28.553445 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a4c2a823-c791-47c6-b9b5-867173385b6b admin admin] The token cache is being invalidate because project c5bf72451ce9448285e7e5d2676200c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:23:28.553739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a4c2a823-c791-47c6-b9b5-867173385b6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:28.554520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a4c2a823-c791-47c6-b9b5-867173385b6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:28.556003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1168/2345] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:23:28 2022] DELETE /identity/v3/projects/c5bf72451ce9448285e7e5d2676200c1 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:23:28.562668 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39993a85-ae68-4e46-97ea-ef5ef8112ccd None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:28.563142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39993a85-ae68-4e46-97ea-ef5ef8112ccd None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:28.563269 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39993a85-ae68-4e46-97ea-ef5ef8112ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:28.582280 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-39993a85-ae68-4e46-97ea-ef5ef8112ccd None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:28.583513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-39993a85-ae68-4e46-97ea-ef5ef8112ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:28.654536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1178/2346] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:28.659537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceaf133-c684-4b0e-9af2-84d59bca9ece None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:28.660046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceaf133-c684-4b0e-9af2-84d59bca9ece None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:28.660408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceaf133-c684-4b0e-9af2-84d59bca9ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:28.685567 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2ceaf133-c684-4b0e-9af2-84d59bca9ece None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:28.687641 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2ceaf133-c684-4b0e-9af2-84d59bca9ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:28.696108 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1169/2347] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:28.700488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39993a85-ae68-4e46-97ea-ef5ef8112ccd None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:28.711522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39993a85-ae68-4e46-97ea-ef5ef8112ccd None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:28.716476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae47136-f7fd-4969-a64e-22c2c3376168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:28.717214 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae47136-f7fd-4969-a64e-22c2c3376168 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:28.717566 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae47136-f7fd-4969-a64e-22c2c3376168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:28.717911 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae47136-f7fd-4969-a64e-22c2c3376168 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:28.718976 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae47136-f7fd-4969-a64e-22c2c3376168 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:23:28.719365 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae47136-f7fd-4969-a64e-22c2c3376168 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:28.721068 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae47136-f7fd-4969-a64e-22c2c3376168 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:28.733217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1179/2348] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:23:28.737665 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceaf133-c684-4b0e-9af2-84d59bca9ece None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:28.746923 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceaf133-c684-4b0e-9af2-84d59bca9ece None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:28.752554 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98e412-4447-4eae-b866-2b0439399384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:28.753211 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98e412-4447-4eae-b866-2b0439399384 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:28.753490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98e412-4447-4eae-b866-2b0439399384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:28.753761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98e412-4447-4eae-b866-2b0439399384 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:28.754533 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98e412-4447-4eae-b866-2b0439399384 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:28.756460 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98e412-4447-4eae-b866-2b0439399384 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:28.835902 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1170/2349] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:23:28 2022] POST /identity/v3/projects => generated 401 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:23:28.840248 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae47136-f7fd-4969-a64e-22c2c3376168 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:28.850339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae47136-f7fd-4969-a64e-22c2c3376168 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:28.908568 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055aaf8-7694-4475-8bff-74b9873873fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:28.909235 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055aaf8-7694-4475-8bff-74b9873873fe admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:28.909523 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055aaf8-7694-4475-8bff-74b9873873fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:28.909805 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055aaf8-7694-4475-8bff-74b9873873fe admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:28.910542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055aaf8-7694-4475-8bff-74b9873873fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:28.912484 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055aaf8-7694-4475-8bff-74b9873873fe admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:29.007741 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:29.012962 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1180/2350] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:23:28 2022] POST /identity/v3/users => generated 301 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:23:29.017352 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98e412-4447-4eae-b866-2b0439399384 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:29.027979 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98e412-4447-4eae-b866-2b0439399384 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:29.035809 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6635b82-0a0e-4d6f-82ee-b9f4d4e72973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:29.036521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6635b82-0a0e-4d6f-82ee-b9f4d4e72973 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:29.040503 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6635b82-0a0e-4d6f-82ee-b9f4d4e72973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:29.040503 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6635b82-0a0e-4d6f-82ee-b9f4d4e72973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:29.040503 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6635b82-0a0e-4d6f-82ee-b9f4d4e72973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:29.040503 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6635b82-0a0e-4d6f-82ee-b9f4d4e72973 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:29.051174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1171/2351] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:23:29.055368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055aaf8-7694-4475-8bff-74b9873873fe admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:29.066497 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055aaf8-7694-4475-8bff-74b9873873fe admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:29.071259 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e266750-9167-46b9-8dab-3c70bd47f6ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:29.071926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e266750-9167-46b9-8dab-3c70bd47f6ca admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:29.072193 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e266750-9167-46b9-8dab-3c70bd47f6ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:29.072470 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e266750-9167-46b9-8dab-3c70bd47f6ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:29.073220 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e266750-9167-46b9-8dab-3c70bd47f6ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:29.074856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e266750-9167-46b9-8dab-3c70bd47f6ca admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:29.085398 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1181/2352] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:23:29.095077 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6635b82-0a0e-4d6f-82ee-b9f4d4e72973 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:29.104990 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6635b82-0a0e-4d6f-82ee-b9f4d4e72973 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:29.110211 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd8a79-905a-47c5-b46e-446c072a89fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:29.111123 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd8a79-905a-47c5-b46e-446c072a89fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:29.111413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd8a79-905a-47c5-b46e-446c072a89fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:29.111691 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd8a79-905a-47c5-b46e-446c072a89fc admin admin] PATH_INFO: `/v3/projects/17ed006e1fc748639ef34b5bf363e48b/users/7f1824e314834713af2359fec6f73368/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:29.126553 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd8a79-905a-47c5-b46e-446c072a89fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=17ed006e1fc748639ef34b5bf363e48b, user_id=7f1824e314834713af2359fec6f73368, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:29.129223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afd8a79-905a-47c5-b46e-446c072a89fc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:29.202823 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1172/2353] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:23:29 2022] PUT /identity/v3/projects/17ed006e1fc748639ef34b5bf363e48b/users/7f1824e314834713af2359fec6f73368/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:23:29.213521 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e090b3-be32-4e06-a582-78064eb49674 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:29.213886 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e090b3-be32-4e06-a582-78064eb49674 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:29.214170 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e090b3-be32-4e06-a582-78064eb49674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:29.241539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e8e090b3-be32-4e06-a582-78064eb49674 None None] MFA Rules not processed for user `7f1824e314834713af2359fec6f73368`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:29.242788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e8e090b3-be32-4e06-a582-78064eb49674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:29.307055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1182/2354] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:29 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:29.311699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd8a79-905a-47c5-b46e-446c072a89fc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:29.373938 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd8a79-905a-47c5-b46e-446c072a89fc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:29.378653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc5df6f-a411-40e5-8573-5a84db95c9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:29.379309 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5df6f-a411-40e5-8573-5a84db95c9d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:29.379593 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5df6f-a411-40e5-8573-5a84db95c9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:29.379870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5df6f-a411-40e5-8573-5a84db95c9d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:29.380573 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc5df6f-a411-40e5-8573-5a84db95c9d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:29.382512 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc5df6f-a411-40e5-8573-5a84db95c9d9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:29.474250 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1173/2355] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:23:29 2022] POST /identity/v3/projects => generated 399 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:23:29.478193 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e090b3-be32-4e06-a582-78064eb49674 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:29.489593 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e090b3-be32-4e06-a582-78064eb49674 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:29.548905 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98841cd6-8acb-4b82-be1b-798ba921b29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:29.549634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98841cd6-8acb-4b82-be1b-798ba921b29c admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:29.549915 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98841cd6-8acb-4b82-be1b-798ba921b29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:29.550192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98841cd6-8acb-4b82-be1b-798ba921b29c admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:29.550917 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98841cd6-8acb-4b82-be1b-798ba921b29c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:29.552845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98841cd6-8acb-4b82-be1b-798ba921b29c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:29.663711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1183/2356] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:23:29 2022] POST /identity/v3/users => generated 306 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:23:29.667416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc5df6f-a411-40e5-8573-5a84db95c9d9 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:29.706805 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc5df6f-a411-40e5-8573-5a84db95c9d9 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:29.714209 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4450dc-15de-42f2-a43f-3921631a9fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:29.715319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4450dc-15de-42f2-a43f-3921631a9fbc admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:29.715785 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4450dc-15de-42f2-a43f-3921631a9fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:29.716306 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4450dc-15de-42f2-a43f-3921631a9fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:29.717548 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4450dc-15de-42f2-a43f-3921631a9fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:29.720332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4450dc-15de-42f2-a43f-3921631a9fbc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:29.739852 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1174/2357] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:23:29.744533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98841cd6-8acb-4b82-be1b-798ba921b29c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:29.759032 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98841cd6-8acb-4b82-be1b-798ba921b29c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:29.766521 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d87d3-a3b9-415c-b935-48ff1ce7dda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:29.767615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d87d3-a3b9-415c-b935-48ff1ce7dda2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:29.771424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d87d3-a3b9-415c-b935-48ff1ce7dda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:29.772273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d87d3-a3b9-415c-b935-48ff1ce7dda2 admin admin] PATH_INFO: `/v3/projects/5d9d623602584337bd5faea1814ac2ea/users/94f89a8ccdb54a7daae5cb7b3609bc56/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:29.811171 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d87d3-a3b9-415c-b935-48ff1ce7dda2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d9d623602584337bd5faea1814ac2ea, user_id=94f89a8ccdb54a7daae5cb7b3609bc56, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:29.824249 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d87d3-a3b9-415c-b935-48ff1ce7dda2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:30.002217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1184/2358] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:23:29 2022] PUT /identity/v3/projects/5d9d623602584337bd5faea1814ac2ea/users/94f89a8ccdb54a7daae5cb7b3609bc56/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:23:30.056783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4450dc-15de-42f2-a43f-3921631a9fbc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:30.074325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4450dc-15de-42f2-a43f-3921631a9fbc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:30.121054 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:30.138883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f5368-c6aa-4866-9f38-3e58ceda597a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:30.139559 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f5368-c6aa-4866-9f38-3e58ceda597a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:30.139924 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f5368-c6aa-4866-9f38-3e58ceda597a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:30.140125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f5368-c6aa-4866-9f38-3e58ceda597a admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:30.140860 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f5368-c6aa-4866-9f38-3e58ceda597a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:30.142561 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f5368-c6aa-4866-9f38-3e58ceda597a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:30.153574 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1175/2359] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:23:30.158217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d87d3-a3b9-415c-b935-48ff1ce7dda2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:30.169863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d87d3-a3b9-415c-b935-48ff1ce7dda2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:30.174825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf595003-f7e9-4d28-af46-f233830c5533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:30.175721 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf595003-f7e9-4d28-af46-f233830c5533 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:30.176341 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf595003-f7e9-4d28-af46-f233830c5533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:30.176769 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf595003-f7e9-4d28-af46-f233830c5533 admin admin] PATH_INFO: `/v3/projects/5d9d623602584337bd5faea1814ac2ea/users/94f89a8ccdb54a7daae5cb7b3609bc56/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:30.179245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf595003-f7e9-4d28-af46-f233830c5533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d9d623602584337bd5faea1814ac2ea, user_id=94f89a8ccdb54a7daae5cb7b3609bc56, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:30.182115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf595003-f7e9-4d28-af46-f233830c5533 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:30.194288 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1185/2360] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:23:30 2022] PUT /identity/v3/projects/5d9d623602584337bd5faea1814ac2ea/users/94f89a8ccdb54a7daae5cb7b3609bc56/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:23:30.206950 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955d823-b8f5-4779-a907-18f2f4e7edf8 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:30.207245 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955d823-b8f5-4779-a907-18f2f4e7edf8 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:30.207532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955d823-b8f5-4779-a907-18f2f4e7edf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:30.233818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e955d823-b8f5-4779-a907-18f2f4e7edf8 None None] MFA Rules not processed for user `94f89a8ccdb54a7daae5cb7b3609bc56`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:30.235064 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e955d823-b8f5-4779-a907-18f2f4e7edf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:30.302582 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1176/2361] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:30 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:23:30.313619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf595003-f7e9-4d28-af46-f233830c5533 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:30.315335 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-cf595003-f7e9-4d28-af46-f233830c5533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:30.350093 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:30.373171 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf595003-f7e9-4d28-af46-f233830c5533 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:30.377082 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263669be-1ffa-4bf9-a37f-8a321c34408c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:30.377781 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263669be-1ffa-4bf9-a37f-8a321c34408c service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:30.378065 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263669be-1ffa-4bf9-a37f-8a321c34408c service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:30.378346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263669be-1ffa-4bf9-a37f-8a321c34408c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:30.387467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263669be-1ffa-4bf9-a37f-8a321c34408c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:30.389599 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263669be-1ffa-4bf9-a37f-8a321c34408c service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:30.399220 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1186/2362] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:23:30 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:23:30.771176 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955d823-b8f5-4779-a907-18f2f4e7edf8 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:30.782797 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955d823-b8f5-4779-a907-18f2f4e7edf8 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:30.786767 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e42689-596a-4d7c-be49-8735b7be2892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:30.787418 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e42689-596a-4d7c-be49-8735b7be2892 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:30.787708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e42689-596a-4d7c-be49-8735b7be2892 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:30.787986 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e42689-596a-4d7c-be49-8735b7be2892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:30.864689 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e42689-596a-4d7c-be49-8735b7be2892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:30.869235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e42689-596a-4d7c-be49-8735b7be2892 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:30.876741 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1177/2363] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:23:30 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:23:32.880809 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde9d81-0c59-4f5f-a01f-217ea1f23bb4 None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:32.881387 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde9d81-0c59-4f5f-a01f-217ea1f23bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:32.881387 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde9d81-0c59-4f5f-a01f-217ea1f23bb4 None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:32.882947 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1187/2364] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jun 24 04:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:23:32.889004 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48ee4a-b5da-49cc-8135-def66e7a9f8c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:32.889494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48ee4a-b5da-49cc-8135-def66e7a9f8c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:32.889596 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc48ee4a-b5da-49cc-8135-def66e7a9f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:32.909912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-fc48ee4a-b5da-49cc-8135-def66e7a9f8c None None] MFA Rules not processed for user `80b7b3ec60db4de19bbf7d08f111f470`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:32.911467 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fc48ee4a-b5da-49cc-8135-def66e7a9f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:32.974244 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:32.987923 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1178/2365] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jun 24 04:23:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:33.000404 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde9d81-0c59-4f5f-a01f-217ea1f23bb4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:33.002235 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1cde9d81-0c59-4f5f-a01f-217ea1f23bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:33.075894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cde9d81-0c59-4f5f-a01f-217ea1f23bb4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:33.079607 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093175cc-5dcd-41be-b200-69610a45903c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61459e15d327405b88a2fdfc1fc0fecd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:33.080419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093175cc-5dcd-41be-b200-69610a45903c service glance] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:33.080624 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093175cc-5dcd-41be-b200-69610a45903c service glance] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:33.080904 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093175cc-5dcd-41be-b200-69610a45903c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:33.094012 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093175cc-5dcd-41be-b200-69610a45903c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:33.095701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093175cc-5dcd-41be-b200-69610a45903c service glance] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:33.108350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1188/2366] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:23:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:23:33.189505 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a24b8-6424-409f-9798-e5b3c413b633 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:33.189799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a24b8-6424-409f-9798-e5b3c413b633 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:33.190084 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a24b8-6424-409f-9798-e5b3c413b633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:33.207860 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a74a24b8-6424-409f-9798-e5b3c413b633 None None] MFA Rules not processed for user `d1cee676e3dc47e18e94de7e20b59455`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:33.209068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a74a24b8-6424-409f-9798-e5b3c413b633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:33.267894 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1179/2367] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:33.339032 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a58b7c-af7e-481e-8835-cb50aecc1bef None None] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:33.339032 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a58b7c-af7e-481e-8835-cb50aecc1bef None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:33.339512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a58b7c-af7e-481e-8835-cb50aecc1bef None None] PATH_INFO: `/` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:33.341722 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1189/2368] 10.222.0.16 () {54 vars in 921 bytes} [Fri Jun 24 04:23:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 04:23:33.347124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073265d-e565-43f0-938b-105833203d2b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:33.347624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073265d-e565-43f0-938b-105833203d2b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:33.347905 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073265d-e565-43f0-938b-105833203d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:33.371324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f073265d-e565-43f0-938b-105833203d2b None None] MFA Rules not processed for user `fd98b54f587f4b42b4dd0a058b30c684`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:33.372879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f073265d-e565-43f0-938b-105833203d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:33.441852 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1180/2369] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Jun 24 04:23:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:33.455825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a58b7c-af7e-481e-8835-cb50aecc1bef None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:33.457844 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-17a58b7c-af7e-481e-8835-cb50aecc1bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:33.472762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a58b7c-af7e-481e-8835-cb50aecc1bef None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:33.476757 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52245dac-e1d5-47b2-bcaa-3af1ba29cd7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:33.477471 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52245dac-e1d5-47b2-bcaa-3af1ba29cd7d service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:33.477764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52245dac-e1d5-47b2-bcaa-3af1ba29cd7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:33.478048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52245dac-e1d5-47b2-bcaa-3af1ba29cd7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:33.486034 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52245dac-e1d5-47b2-bcaa-3af1ba29cd7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:33.487812 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52245dac-e1d5-47b2-bcaa-3af1ba29cd7d service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:33.495768 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1190/2370] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:23:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:23:36.041026 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42857b0-e455-49e2-af42-3c0bfe66590d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:36.046639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42857b0-e455-49e2-af42-3c0bfe66590d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:36.046639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42857b0-e455-49e2-af42-3c0bfe66590d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:36.062334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d42857b0-e455-49e2-af42-3c0bfe66590d None None] MFA Rules not processed for user `98f58f91d17949198e66883d234a9e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:36.063624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d42857b0-e455-49e2-af42-3c0bfe66590d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:36.123103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1181/2371] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Jun 24 04:23:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:36.135021 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52245dac-e1d5-47b2-bcaa-3af1ba29cd7d service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:36.136636 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-52245dac-e1d5-47b2-bcaa-3af1ba29cd7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:36.183496 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42857b0-e455-49e2-af42-3c0bfe66590d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:36.213460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52245dac-e1d5-47b2-bcaa-3af1ba29cd7d service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:36.219233 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc291ef-0c6e-4948-a05c-dc6bd295e231 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8e1f1e3c7e42cc9fa6f82aaac55129', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:36.219854 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc291ef-0c6e-4948-a05c-dc6bd295e231 service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:36.220138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc291ef-0c6e-4948-a05c-dc6bd295e231 service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:36.220420 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc291ef-0c6e-4948-a05c-dc6bd295e231 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:36.229049 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc291ef-0c6e-4948-a05c-dc6bd295e231 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:36.230753 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc291ef-0c6e-4948-a05c-dc6bd295e231 service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:36.239463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1191/2372] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jun 24 04:23:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 04:23:36.245713 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:36.253044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42857b0-e455-49e2-af42-3c0bfe66590d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:36.257614 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b48fa-a38b-4ada-8c41-78f5dc8b1dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:36.258518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b48fa-a38b-4ada-8c41-78f5dc8b1dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:36.258803 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b48fa-a38b-4ada-8c41-78f5dc8b1dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:36.259074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b48fa-a38b-4ada-8c41-78f5dc8b1dad admin admin] PATH_INFO: `/v3/users/7f1824e314834713af2359fec6f73368` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:36.260276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b48fa-a38b-4ada-8c41-78f5dc8b1dad admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f1824e314834713af2359fec6f73368)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:36.262232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b48fa-a38b-4ada-8c41-78f5dc8b1dad admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:36.514997 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c82b48fa-a38b-4ada-8c41-78f5dc8b1dad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f1824e314834713af2359fec6f73368'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:36.533858 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c82b48fa-a38b-4ada-8c41-78f5dc8b1dad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f1824e314834713af2359fec6f73368'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:36.612010 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c82b48fa-a38b-4ada-8c41-78f5dc8b1dad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f1824e314834713af2359fec6f73368'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:36.612771 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c82b48fa-a38b-4ada-8c41-78f5dc8b1dad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f1824e314834713af2359fec6f73368'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:36.613508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c82b48fa-a38b-4ada-8c41-78f5dc8b1dad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f1824e314834713af2359fec6f73368'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:36.652463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1182/2373] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:23:36 2022] DELETE /identity/v3/users/7f1824e314834713af2359fec6f73368 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:23:36.658340 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1598aae-2d42-42c4-ad9f-512aac4fb042 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:36.658799 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1598aae-2d42-42c4-ad9f-512aac4fb042 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:36.659261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1598aae-2d42-42c4-ad9f-512aac4fb042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:36.699188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e1598aae-2d42-42c4-ad9f-512aac4fb042 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:36.701428 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e1598aae-2d42-42c4-ad9f-512aac4fb042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:36.799720 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:36.817380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1192/2374] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:36.831433 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b48fa-a38b-4ada-8c41-78f5dc8b1dad admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:36.833243 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c82b48fa-a38b-4ada-8c41-78f5dc8b1dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:36.929705 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b48fa-a38b-4ada-8c41-78f5dc8b1dad admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:36.936356 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85ed66-fd1e-4095-ade7-33aabbad1908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:36.937111 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85ed66-fd1e-4095-ade7-33aabbad1908 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:36.937729 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85ed66-fd1e-4095-ade7-33aabbad1908 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:36.938017 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85ed66-fd1e-4095-ade7-33aabbad1908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:36.951640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85ed66-fd1e-4095-ade7-33aabbad1908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:36.954162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85ed66-fd1e-4095-ade7-33aabbad1908 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:36.969699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1183/2375] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:23:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:23:37.468134 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1598aae-2d42-42c4-ad9f-512aac4fb042 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:37.469706 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e1598aae-2d42-42c4-ad9f-512aac4fb042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:37.489068 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1598aae-2d42-42c4-ad9f-512aac4fb042 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:37.501673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fdccf-cb92-43fe-8c01-90ad1f85ea41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:37.502680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fdccf-cb92-43fe-8c01-90ad1f85ea41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:37.502967 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fdccf-cb92-43fe-8c01-90ad1f85ea41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:37.503244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fdccf-cb92-43fe-8c01-90ad1f85ea41 admin admin] PATH_INFO: `/v3/projects/17ed006e1fc748639ef34b5bf363e48b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:37.504489 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fdccf-cb92-43fe-8c01-90ad1f85ea41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17ed006e1fc748639ef34b5bf363e48b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:37.506564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fdccf-cb92-43fe-8c01-90ad1f85ea41 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:37.631895 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-329fdccf-cb92-43fe-8c01-90ad1f85ea41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17ed006e1fc748639ef34b5bf363e48b', 'request_id': 'req-329fdccf-cb92-43fe-8c01-90ad1f85ea41'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:37.645913 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-329fdccf-cb92-43fe-8c01-90ad1f85ea41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17ed006e1fc748639ef34b5bf363e48b', 'request_id': 'req-329fdccf-cb92-43fe-8c01-90ad1f85ea41'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:37.686957 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-329fdccf-cb92-43fe-8c01-90ad1f85ea41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17ed006e1fc748639ef34b5bf363e48b', 'request_id': 'req-329fdccf-cb92-43fe-8c01-90ad1f85ea41'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:37.773870 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-329fdccf-cb92-43fe-8c01-90ad1f85ea41 admin admin] The token cache is being invalidate because project 17ed006e1fc748639ef34b5bf363e48b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:23:37.774701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-329fdccf-cb92-43fe-8c01-90ad1f85ea41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:37.777257 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-329fdccf-cb92-43fe-8c01-90ad1f85ea41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:37.781668 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1193/2376] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:23:37 2022] DELETE /identity/v3/projects/17ed006e1fc748639ef34b5bf363e48b => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:23:37.783321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85ed66-fd1e-4095-ade7-33aabbad1908 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:37.810150 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85ed66-fd1e-4095-ade7-33aabbad1908 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:37.871873 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8439b-df1e-45a3-b03c-96cde5fb3b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:37.873302 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8439b-df1e-45a3-b03c-96cde5fb3b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:37.873645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8439b-df1e-45a3-b03c-96cde5fb3b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:37.873932 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8439b-df1e-45a3-b03c-96cde5fb3b5a admin admin] PATH_INFO: `/v3/users/94f89a8ccdb54a7daae5cb7b3609bc56` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:37.875257 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8439b-df1e-45a3-b03c-96cde5fb3b5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=94f89a8ccdb54a7daae5cb7b3609bc56)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:37.877230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8439b-df1e-45a3-b03c-96cde5fb3b5a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:37.900123 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:38.173639 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f3a8439b-df1e-45a3-b03c-96cde5fb3b5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94f89a8ccdb54a7daae5cb7b3609bc56'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:38.186730 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f3a8439b-df1e-45a3-b03c-96cde5fb3b5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94f89a8ccdb54a7daae5cb7b3609bc56'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:38.269723 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f3a8439b-df1e-45a3-b03c-96cde5fb3b5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94f89a8ccdb54a7daae5cb7b3609bc56'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:38.270388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f3a8439b-df1e-45a3-b03c-96cde5fb3b5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94f89a8ccdb54a7daae5cb7b3609bc56'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:38.271210 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f3a8439b-df1e-45a3-b03c-96cde5fb3b5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94f89a8ccdb54a7daae5cb7b3609bc56'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:38.290542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1184/2377] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:23:37 2022] DELETE /identity/v3/users/94f89a8ccdb54a7daae5cb7b3609bc56 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:23:38.788390 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fdccf-cb92-43fe-8c01-90ad1f85ea41 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:38.789878 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-329fdccf-cb92-43fe-8c01-90ad1f85ea41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:38.804028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fdccf-cb92-43fe-8c01-90ad1f85ea41 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:38.895926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c40462-ffea-4829-a565-3c93386c0751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:38.897318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c40462-ffea-4829-a565-3c93386c0751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:38.897844 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c40462-ffea-4829-a565-3c93386c0751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:38.898325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c40462-ffea-4829-a565-3c93386c0751 admin admin] PATH_INFO: `/v3/projects/5d9d623602584337bd5faea1814ac2ea` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:38.900258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c40462-ffea-4829-a565-3c93386c0751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d9d623602584337bd5faea1814ac2ea)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:38.902583 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c40462-ffea-4829-a565-3c93386c0751 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:39.127968 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-85c40462-ffea-4829-a565-3c93386c0751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d9d623602584337bd5faea1814ac2ea', 'request_id': 'req-85c40462-ffea-4829-a565-3c93386c0751'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:39.134615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-85c40462-ffea-4829-a565-3c93386c0751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d9d623602584337bd5faea1814ac2ea', 'request_id': 'req-85c40462-ffea-4829-a565-3c93386c0751'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:39.140844 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-85c40462-ffea-4829-a565-3c93386c0751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d9d623602584337bd5faea1814ac2ea', 'request_id': 'req-85c40462-ffea-4829-a565-3c93386c0751'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:39.230883 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-85c40462-ffea-4829-a565-3c93386c0751 admin admin] The token cache is being invalidate because project 5d9d623602584337bd5faea1814ac2ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:23:39.231245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-85c40462-ffea-4829-a565-3c93386c0751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:39.232636 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-85c40462-ffea-4829-a565-3c93386c0751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:39.234249 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1194/2378] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:23:38 2022] DELETE /identity/v3/projects/5d9d623602584337bd5faea1814ac2ea => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:23:39.239391 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05f476-1ed4-4ad8-9c06-019e1a443e6c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:39.239992 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05f476-1ed4-4ad8-9c06-019e1a443e6c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:39.239992 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05f476-1ed4-4ad8-9c06-019e1a443e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:39.275524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-af05f476-1ed4-4ad8-9c06-019e1a443e6c None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:39.280715 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-af05f476-1ed4-4ad8-9c06-019e1a443e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:39.346662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1185/2379] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:39.352714 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e23cb-136b-48e4-82f4-b37b0531bcd4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:39.353007 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e23cb-136b-48e4-82f4-b37b0531bcd4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:39.353290 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167e23cb-136b-48e4-82f4-b37b0531bcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:39.372006 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:39.377835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-167e23cb-136b-48e4-82f4-b37b0531bcd4 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:39.379111 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-167e23cb-136b-48e4-82f4-b37b0531bcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:39.392354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1195/2380] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:39.396780 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05f476-1ed4-4ad8-9c06-019e1a443e6c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:39.407813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05f476-1ed4-4ad8-9c06-019e1a443e6c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:39.412886 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1cf8b-eeec-42d0-8d56-4ecd2304ecb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:39.413644 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1cf8b-eeec-42d0-8d56-4ecd2304ecb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:39.414031 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1cf8b-eeec-42d0-8d56-4ecd2304ecb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:39.414441 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c1cf8b-eeec-42d0-8d56-4ecd2304ecb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:39.415242 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1cf8b-eeec-42d0-8d56-4ecd2304ecb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:23:39.415619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1cf8b-eeec-42d0-8d56-4ecd2304ecb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:39.417291 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c1cf8b-eeec-42d0-8d56-4ecd2304ecb4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:39.431853 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1186/2381] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:23:39.436647 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e23cb-136b-48e4-82f4-b37b0531bcd4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:39.446245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167e23cb-136b-48e4-82f4-b37b0531bcd4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:39.451520 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f28fb8-d65a-4b23-8ec7-bba842379217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:39.452429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f28fb8-d65a-4b23-8ec7-bba842379217 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:39.452536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f28fb8-d65a-4b23-8ec7-bba842379217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:39.452730 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f28fb8-d65a-4b23-8ec7-bba842379217 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:39.453473 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f28fb8-d65a-4b23-8ec7-bba842379217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:39.455407 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f28fb8-d65a-4b23-8ec7-bba842379217 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:39.567560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1196/2382] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:23:39 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:23:39.571314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1cf8b-eeec-42d0-8d56-4ecd2304ecb4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:39.581518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c1cf8b-eeec-42d0-8d56-4ecd2304ecb4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:39.645319 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42d285-37f8-49fd-b521-409a62dc62af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:39.646008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42d285-37f8-49fd-b521-409a62dc62af admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:39.646292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42d285-37f8-49fd-b521-409a62dc62af admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:39.646612 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee42d285-37f8-49fd-b521-409a62dc62af admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:39.647323 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42d285-37f8-49fd-b521-409a62dc62af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:39.649515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee42d285-37f8-49fd-b521-409a62dc62af admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:39.746647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1187/2383] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:23:39 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:23:39.751120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f28fb8-d65a-4b23-8ec7-bba842379217 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:39.775565 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f28fb8-d65a-4b23-8ec7-bba842379217 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:39.781672 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e6f65-3ca2-4309-aaae-ffe4df7d8c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:39.782449 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e6f65-3ca2-4309-aaae-ffe4df7d8c66 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:39.782697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e6f65-3ca2-4309-aaae-ffe4df7d8c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:39.783003 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e6f65-3ca2-4309-aaae-ffe4df7d8c66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:39.783861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e6f65-3ca2-4309-aaae-ffe4df7d8c66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:39.785461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e6f65-3ca2-4309-aaae-ffe4df7d8c66 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:39.796536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1197/2384] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:23:39.800578 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42d285-37f8-49fd-b521-409a62dc62af admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:39.811609 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee42d285-37f8-49fd-b521-409a62dc62af admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:39.819855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75feca5e-728a-4ca1-a4e5-9019ef679705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:39.819855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75feca5e-728a-4ca1-a4e5-9019ef679705 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:39.819855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75feca5e-728a-4ca1-a4e5-9019ef679705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:39.819855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75feca5e-728a-4ca1-a4e5-9019ef679705 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:39.820414 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75feca5e-728a-4ca1-a4e5-9019ef679705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:39.820414 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75feca5e-728a-4ca1-a4e5-9019ef679705 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:39.834815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1188/2385] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:23:39.836508 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e6f65-3ca2-4309-aaae-ffe4df7d8c66 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:39.847829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e6f65-3ca2-4309-aaae-ffe4df7d8c66 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:39.854348 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c64d15-a42b-4aa3-b094-a41e63b5b11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:39.854348 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c64d15-a42b-4aa3-b094-a41e63b5b11d admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:39.854348 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c64d15-a42b-4aa3-b094-a41e63b5b11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:39.854348 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c64d15-a42b-4aa3-b094-a41e63b5b11d admin admin] PATH_INFO: `/v3/projects/01195f01757a4da582990dff38ecc93c/users/5f48618d1c4e44eaaf2c04c0a02d2d01/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:39.868977 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c64d15-a42b-4aa3-b094-a41e63b5b11d admin admin] RBAC: Authorizing `identity:create_grant(project_id=01195f01757a4da582990dff38ecc93c, user_id=5f48618d1c4e44eaaf2c04c0a02d2d01, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:39.871722 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c64d15-a42b-4aa3-b094-a41e63b5b11d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:39.957962 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1198/2386] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:23:39 2022] PUT /identity/v3/projects/01195f01757a4da582990dff38ecc93c/users/5f48618d1c4e44eaaf2c04c0a02d2d01/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:23:39.968383 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07844b3d-1606-4120-ab8e-8f429f7f88e0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:39.968671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07844b3d-1606-4120-ab8e-8f429f7f88e0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:39.968952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07844b3d-1606-4120-ab8e-8f429f7f88e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:39.974877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:40.025137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-07844b3d-1606-4120-ab8e-8f429f7f88e0 None None] MFA Rules not processed for user `5f48618d1c4e44eaaf2c04c0a02d2d01`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:40.026357 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-07844b3d-1606-4120-ab8e-8f429f7f88e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:40.130406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1189/2387] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:40.142079 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c64d15-a42b-4aa3-b094-a41e63b5b11d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:40.143885 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b9c64d15-a42b-4aa3-b094-a41e63b5b11d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:40.219954 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c64d15-a42b-4aa3-b094-a41e63b5b11d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:40.229141 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c6d18-ed29-4fdc-8cd6-26e95cd9e6ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:40.229894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c6d18-ed29-4fdc-8cd6-26e95cd9e6ac service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:40.230099 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c6d18-ed29-4fdc-8cd6-26e95cd9e6ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:40.230406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c6d18-ed29-4fdc-8cd6-26e95cd9e6ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:40.254665 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c6d18-ed29-4fdc-8cd6-26e95cd9e6ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:40.258525 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c6d18-ed29-4fdc-8cd6-26e95cd9e6ac service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:40.271376 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1199/2388] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:23:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:23:40.293078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07844b3d-1606-4120-ab8e-8f429f7f88e0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:40.377388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07844b3d-1606-4120-ab8e-8f429f7f88e0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:40.382862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62c4c2-9673-4496-b366-d02de2adf1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:40.383758 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62c4c2-9673-4496-b366-d02de2adf1f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:40.384047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62c4c2-9673-4496-b366-d02de2adf1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:40.384327 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62c4c2-9673-4496-b366-d02de2adf1f7 admin admin] PATH_INFO: `/v3/users/5f48618d1c4e44eaaf2c04c0a02d2d01` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:40.385560 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62c4c2-9673-4496-b366-d02de2adf1f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f48618d1c4e44eaaf2c04c0a02d2d01)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:40.388092 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62c4c2-9673-4496-b366-d02de2adf1f7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:40.558506 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0d62c4c2-9673-4496-b366-d02de2adf1f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f48618d1c4e44eaaf2c04c0a02d2d01'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:40.574882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0d62c4c2-9673-4496-b366-d02de2adf1f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f48618d1c4e44eaaf2c04c0a02d2d01'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:40.635397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0d62c4c2-9673-4496-b366-d02de2adf1f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f48618d1c4e44eaaf2c04c0a02d2d01'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:40.636229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0d62c4c2-9673-4496-b366-d02de2adf1f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f48618d1c4e44eaaf2c04c0a02d2d01'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:40.637106 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0d62c4c2-9673-4496-b366-d02de2adf1f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f48618d1c4e44eaaf2c04c0a02d2d01'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:40.654467 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1190/2389] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:23:40 2022] DELETE /identity/v3/users/5f48618d1c4e44eaaf2c04c0a02d2d01 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:23:40.659178 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e9f9d-43ff-42b5-b594-292021dc666b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:40.659560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e9f9d-43ff-42b5-b594-292021dc666b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:40.659755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e9f9d-43ff-42b5-b594-292021dc666b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:40.677164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-900e9f9d-43ff-42b5-b594-292021dc666b None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:40.678383 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-900e9f9d-43ff-42b5-b594-292021dc666b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:40.751287 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1200/2390] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:40.766667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62c4c2-9673-4496-b366-d02de2adf1f7 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:40.768382 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0d62c4c2-9673-4496-b366-d02de2adf1f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:40.788520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:40.836543 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62c4c2-9673-4496-b366-d02de2adf1f7 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:40.841415 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598c43e-c2a1-412a-8108-02f38c3cd621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:40.842278 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598c43e-c2a1-412a-8108-02f38c3cd621 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:40.842594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598c43e-c2a1-412a-8108-02f38c3cd621 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:40.842881 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598c43e-c2a1-412a-8108-02f38c3cd621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:40.853706 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598c43e-c2a1-412a-8108-02f38c3cd621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:40.855202 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598c43e-c2a1-412a-8108-02f38c3cd621 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:40.866693 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1191/2391] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:23:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:23:41.409816 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e9f9d-43ff-42b5-b594-292021dc666b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:41.411368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-900e9f9d-43ff-42b5-b594-292021dc666b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:41.425921 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e9f9d-43ff-42b5-b594-292021dc666b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:41.431921 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4187a-3ecb-456d-9304-ef0edb916452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:41.432797 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4187a-3ecb-456d-9304-ef0edb916452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:41.433133 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4187a-3ecb-456d-9304-ef0edb916452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:41.433419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4187a-3ecb-456d-9304-ef0edb916452 admin admin] PATH_INFO: `/v3/projects/01195f01757a4da582990dff38ecc93c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:41.435345 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d4187a-3ecb-456d-9304-ef0edb916452 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01195f01757a4da582990dff38ecc93c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:41.437291 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d4187a-3ecb-456d-9304-ef0edb916452 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:41.585745 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:41.598896 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d6d4187a-3ecb-456d-9304-ef0edb916452 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01195f01757a4da582990dff38ecc93c', 'request_id': 'req-d6d4187a-3ecb-456d-9304-ef0edb916452'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:41.607034 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d6d4187a-3ecb-456d-9304-ef0edb916452 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01195f01757a4da582990dff38ecc93c', 'request_id': 'req-d6d4187a-3ecb-456d-9304-ef0edb916452'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:41.613724 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d6d4187a-3ecb-456d-9304-ef0edb916452 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01195f01757a4da582990dff38ecc93c', 'request_id': 'req-d6d4187a-3ecb-456d-9304-ef0edb916452'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:41.697969 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d6d4187a-3ecb-456d-9304-ef0edb916452 admin admin] The token cache is being invalidate because project 01195f01757a4da582990dff38ecc93c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:23:41.698336 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d6d4187a-3ecb-456d-9304-ef0edb916452 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:41.702263 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d6d4187a-3ecb-456d-9304-ef0edb916452 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:23:41.705008 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1201/2392] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:23:41 2022] DELETE /identity/v3/projects/01195f01757a4da582990dff38ecc93c => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:23:41.709695 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12945621-540a-4855-ada3-a6b57cbad514 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:41.710157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12945621-540a-4855-ada3-a6b57cbad514 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:41.710239 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12945621-540a-4855-ada3-a6b57cbad514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:41.729987 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-12945621-540a-4855-ada3-a6b57cbad514 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:41.732095 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-12945621-540a-4855-ada3-a6b57cbad514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:41.807320 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1192/2393] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:41.820960 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac3444-b3f4-4c84-933c-b2ab11664b72 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:41.821264 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac3444-b3f4-4c84-933c-b2ab11664b72 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:41.824648 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac3444-b3f4-4c84-933c-b2ab11664b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:41.844215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f8ac3444-b3f4-4c84-933c-b2ab11664b72 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:41.845430 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f8ac3444-b3f4-4c84-933c-b2ab11664b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:41.861889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1202/2394] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:41.862507 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12945621-540a-4855-ada3-a6b57cbad514 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:41.873730 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12945621-540a-4855-ada3-a6b57cbad514 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:41.878732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5179c1f-37d6-414e-b7be-d4b6faf0b542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:41.885292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5179c1f-37d6-414e-b7be-d4b6faf0b542 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:41.885292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5179c1f-37d6-414e-b7be-d4b6faf0b542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:41.885292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5179c1f-37d6-414e-b7be-d4b6faf0b542 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:41.885292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5179c1f-37d6-414e-b7be-d4b6faf0b542 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:23:41.885292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5179c1f-37d6-414e-b7be-d4b6faf0b542 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:41.885292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5179c1f-37d6-414e-b7be-d4b6faf0b542 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:41.896793 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1193/2395] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:23:41.901632 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac3444-b3f4-4c84-933c-b2ab11664b72 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:41.911982 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac3444-b3f4-4c84-933c-b2ab11664b72 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:41.924358 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65693f7-4217-4cea-a189-c31018adef16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:41.925312 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65693f7-4217-4cea-a189-c31018adef16 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:41.925475 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65693f7-4217-4cea-a189-c31018adef16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:41.925777 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65693f7-4217-4cea-a189-c31018adef16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:41.926516 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65693f7-4217-4cea-a189-c31018adef16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:41.928574 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65693f7-4217-4cea-a189-c31018adef16 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:42.040903 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1203/2396] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:23:41 2022] POST /identity/v3/projects => generated 377 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:23:42.042191 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5179c1f-37d6-414e-b7be-d4b6faf0b542 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:42.052688 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5179c1f-37d6-414e-b7be-d4b6faf0b542 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:42.114465 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5eb6c1-7c0e-49ee-8baf-fb01497804b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:42.115135 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eb6c1-7c0e-49ee-8baf-fb01497804b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:42.115499 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eb6c1-7c0e-49ee-8baf-fb01497804b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:42.115845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eb6c1-7c0e-49ee-8baf-fb01497804b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:42.116581 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5eb6c1-7c0e-49ee-8baf-fb01497804b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:42.118593 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5eb6c1-7c0e-49ee-8baf-fb01497804b0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:42.204665 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1194/2397] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:23:42 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:23:42.208625 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65693f7-4217-4cea-a189-c31018adef16 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:42.220552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65693f7-4217-4cea-a189-c31018adef16 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:42.225432 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a8995-e013-403b-9a4f-1d13273ae428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:42.226154 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a8995-e013-403b-9a4f-1d13273ae428 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:42.226465 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a8995-e013-403b-9a4f-1d13273ae428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:42.226749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5a8995-e013-403b-9a4f-1d13273ae428 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:42.227659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a8995-e013-403b-9a4f-1d13273ae428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:42.229322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5a8995-e013-403b-9a4f-1d13273ae428 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:42.241298 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1204/2398] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:23:42.245374 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5eb6c1-7c0e-49ee-8baf-fb01497804b0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:42.258446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5eb6c1-7c0e-49ee-8baf-fb01497804b0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:42.268899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586df1b2-f7de-46d8-84ca-f1e101446eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:42.268899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586df1b2-f7de-46d8-84ca-f1e101446eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:42.268899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586df1b2-f7de-46d8-84ca-f1e101446eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:42.268899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586df1b2-f7de-46d8-84ca-f1e101446eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:42.269210 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586df1b2-f7de-46d8-84ca-f1e101446eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:42.269210 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586df1b2-f7de-46d8-84ca-f1e101446eb8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:42.282416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1195/2399] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:23:42.284900 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a8995-e013-403b-9a4f-1d13273ae428 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:42.296601 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5a8995-e013-403b-9a4f-1d13273ae428 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:42.301695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2903a-92ea-40d3-9091-7f17dfcfcc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:42.302870 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2903a-92ea-40d3-9091-7f17dfcfcc28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:42.303022 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2903a-92ea-40d3-9091-7f17dfcfcc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:42.303211 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2903a-92ea-40d3-9091-7f17dfcfcc28 admin admin] PATH_INFO: `/v3/projects/858c133aca394db2a5e8bbef6ad3e6b2/users/f6e382ff09f043518deee014679412d6/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:42.321955 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e2903a-92ea-40d3-9091-7f17dfcfcc28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=858c133aca394db2a5e8bbef6ad3e6b2, user_id=f6e382ff09f043518deee014679412d6, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:42.326724 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e2903a-92ea-40d3-9091-7f17dfcfcc28 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:42.427422 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1205/2400] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:23:42 2022] PUT /identity/v3/projects/858c133aca394db2a5e8bbef6ad3e6b2/users/f6e382ff09f043518deee014679412d6/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:23:42.435488 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b946e60-c596-4db3-a10b-c9baa2e6e61a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:42.436265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b946e60-c596-4db3-a10b-c9baa2e6e61a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:42.436798 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b946e60-c596-4db3-a10b-c9baa2e6e61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:42.474055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9b946e60-c596-4db3-a10b-c9baa2e6e61a None None] MFA Rules not processed for user `f6e382ff09f043518deee014679412d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:42.475333 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9b946e60-c596-4db3-a10b-c9baa2e6e61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:42.484703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:42.571857 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1196/2401] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:23:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:42.581300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2903a-92ea-40d3-9091-7f17dfcfcc28 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:42.583540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-98e2903a-92ea-40d3-9091-7f17dfcfcc28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:42.693394 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2903a-92ea-40d3-9091-7f17dfcfcc28 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:42.698599 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b90915-fa85-4b23-b4d2-426fe6f3253d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:42.699259 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b90915-fa85-4b23-b4d2-426fe6f3253d service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:42.699551 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b90915-fa85-4b23-b4d2-426fe6f3253d service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:42.699829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b90915-fa85-4b23-b4d2-426fe6f3253d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:42.721628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b90915-fa85-4b23-b4d2-426fe6f3253d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:42.721628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b90915-fa85-4b23-b4d2-426fe6f3253d service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:42.731685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1206/2402] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:23:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:23:52.573943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=23 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:52.726010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:23:57.913506 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66134e-26ce-4a38-838b-96dc1d80f507 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:57.914030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66134e-26ce-4a38-838b-96dc1d80f507 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:57.914030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf66134e-26ce-4a38-838b-96dc1d80f507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:57.935185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-bf66134e-26ce-4a38-838b-96dc1d80f507 None None] MFA Rules not processed for user `98f58f91d17949198e66883d234a9e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:23:57.936406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bf66134e-26ce-4a38-838b-96dc1d80f507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:58.008938 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1197/2403] 10.222.0.16 () {58 vars in 967 bytes} [Fri Jun 24 04:23:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:23:58.070097 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b90915-fa85-4b23-b4d2-426fe6f3253d service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:23:58.072020 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-10b90915-fa85-4b23-b4d2-426fe6f3253d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:23:58.182678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b90915-fa85-4b23-b4d2-426fe6f3253d service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:23:58.196684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc91ac81-c316-45cc-8115-765611751488 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d8e1f1e3c7e42cc9fa6f82aaac55129', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:23:58.197569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91ac81-c316-45cc-8115-765611751488 service swift] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:23:58.197931 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91ac81-c316-45cc-8115-765611751488 service swift] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:23:58.198288 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91ac81-c316-45cc-8115-765611751488 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:23:58.208772 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91ac81-c316-45cc-8115-765611751488 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:23:58.210792 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc91ac81-c316-45cc-8115-765611751488 service swift] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:23:58.220016 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1207/2404] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Jun 24 04:23:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 04:24:08.001845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=25 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:08.216394 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=32 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:09.300829 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66134e-26ce-4a38-838b-96dc1d80f507 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:09.365601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf66134e-26ce-4a38-838b-96dc1d80f507 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:09.370194 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206e6cf-3610-4d48-81bc-dc20d272bb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:09.371102 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206e6cf-3610-4d48-81bc-dc20d272bb6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:09.371435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206e6cf-3610-4d48-81bc-dc20d272bb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:09.371718 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206e6cf-3610-4d48-81bc-dc20d272bb6f admin admin] PATH_INFO: `/v3/users/f6e382ff09f043518deee014679412d6` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:09.372915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206e6cf-3610-4d48-81bc-dc20d272bb6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6e382ff09f043518deee014679412d6)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:09.374883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206e6cf-3610-4d48-81bc-dc20d272bb6f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:09.557738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e206e6cf-3610-4d48-81bc-dc20d272bb6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6e382ff09f043518deee014679412d6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:09.569281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e206e6cf-3610-4d48-81bc-dc20d272bb6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6e382ff09f043518deee014679412d6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:09.630051 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e206e6cf-3610-4d48-81bc-dc20d272bb6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6e382ff09f043518deee014679412d6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:09.630754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e206e6cf-3610-4d48-81bc-dc20d272bb6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6e382ff09f043518deee014679412d6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:09.631433 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e206e6cf-3610-4d48-81bc-dc20d272bb6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6e382ff09f043518deee014679412d6'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:09.645061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1198/2405] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:24:09 2022] DELETE /identity/v3/users/f6e382ff09f043518deee014679412d6 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:09.649936 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065020a-88f6-4f3c-9919-cab65db48388 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:09.650349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065020a-88f6-4f3c-9919-cab65db48388 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:09.650541 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4065020a-88f6-4f3c-9919-cab65db48388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:09.668753 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4065020a-88f6-4f3c-9919-cab65db48388 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:09.669950 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4065020a-88f6-4f3c-9919-cab65db48388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:09.736622 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1208/2406] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:09.746824 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206e6cf-3610-4d48-81bc-dc20d272bb6f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:09.748390 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e206e6cf-3610-4d48-81bc-dc20d272bb6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:09.818097 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206e6cf-3610-4d48-81bc-dc20d272bb6f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:09.821956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981c08f-a538-4684-b611-4b477833cc8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:09.823452 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981c08f-a538-4684-b611-4b477833cc8d service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:09.823756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981c08f-a538-4684-b611-4b477833cc8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:09.824021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981c08f-a538-4684-b611-4b477833cc8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:09.834649 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981c08f-a538-4684-b611-4b477833cc8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:09.839219 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981c08f-a538-4684-b611-4b477833cc8d service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:09.847402 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1199/2407] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:24:10.049843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065020a-88f6-4f3c-9919-cab65db48388 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:10.059409 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4065020a-88f6-4f3c-9919-cab65db48388 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:10.063446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8950773-45c6-4d30-83a6-d3124dc1aba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:10.073548 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8950773-45c6-4d30-83a6-d3124dc1aba8 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:10.073548 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8950773-45c6-4d30-83a6-d3124dc1aba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:10.073548 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8950773-45c6-4d30-83a6-d3124dc1aba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:10.073548 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a8950773-45c6-4d30-83a6-d3124dc1aba8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:10.133404 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8950773-45c6-4d30-83a6-d3124dc1aba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:10.134885 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8950773-45c6-4d30-83a6-d3124dc1aba8 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:10.154427 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1209/2408] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:24:10 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:24:10.370334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981c08f-a538-4684-b611-4b477833cc8d service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:10.371883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e981c08f-a538-4684-b611-4b477833cc8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:10.384886 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981c08f-a538-4684-b611-4b477833cc8d service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:10.390719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22942c9d-981f-44c7-82d8-95513f6f7dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:10.392861 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22942c9d-981f-44c7-82d8-95513f6f7dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:10.393172 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22942c9d-981f-44c7-82d8-95513f6f7dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:10.393454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22942c9d-981f-44c7-82d8-95513f6f7dea admin admin] PATH_INFO: `/v3/projects/858c133aca394db2a5e8bbef6ad3e6b2` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:10.395524 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22942c9d-981f-44c7-82d8-95513f6f7dea admin admin] RBAC: Authorizing `identity:delete_project(project_id=858c133aca394db2a5e8bbef6ad3e6b2)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:10.398464 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22942c9d-981f-44c7-82d8-95513f6f7dea admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:10.435046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:10.540441 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-22942c9d-981f-44c7-82d8-95513f6f7dea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '858c133aca394db2a5e8bbef6ad3e6b2', 'request_id': 'req-22942c9d-981f-44c7-82d8-95513f6f7dea'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:10.629101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-22942c9d-981f-44c7-82d8-95513f6f7dea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '858c133aca394db2a5e8bbef6ad3e6b2', 'request_id': 'req-22942c9d-981f-44c7-82d8-95513f6f7dea'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:10.636140 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-22942c9d-981f-44c7-82d8-95513f6f7dea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '858c133aca394db2a5e8bbef6ad3e6b2', 'request_id': 'req-22942c9d-981f-44c7-82d8-95513f6f7dea'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:10.648571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-22942c9d-981f-44c7-82d8-95513f6f7dea admin admin] The token cache is being invalidate because project 858c133aca394db2a5e8bbef6ad3e6b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:24:10.648952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-22942c9d-981f-44c7-82d8-95513f6f7dea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:10.649924 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-22942c9d-981f-44c7-82d8-95513f6f7dea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:10.651543 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1200/2409] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:24:10 2022] DELETE /identity/v3/projects/858c133aca394db2a5e8bbef6ad3e6b2 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:10.656729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f2825-8b10-40b5-9a0f-257f3ecd4c29 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:10.657653 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f2825-8b10-40b5-9a0f-257f3ecd4c29 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:10.658006 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f2825-8b10-40b5-9a0f-257f3ecd4c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:10.677198 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-ea8f2825-8b10-40b5-9a0f-257f3ecd4c29 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:10.678570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ea8f2825-8b10-40b5-9a0f-257f3ecd4c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:10.746482 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1210/2410] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:10.754827 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11e798-3894-47e7-aaaa-824838ac72f7 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:10.755117 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11e798-3894-47e7-aaaa-824838ac72f7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:10.755399 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11e798-3894-47e7-aaaa-824838ac72f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:10.775890 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4c11e798-3894-47e7-aaaa-824838ac72f7 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:10.777245 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4c11e798-3894-47e7-aaaa-824838ac72f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:10.786600 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1201/2411] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:10.790870 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f2825-8b10-40b5-9a0f-257f3ecd4c29 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:10.803061 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f2825-8b10-40b5-9a0f-257f3ecd4c29 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:10.808149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823411db-d95b-4209-a6b6-f755cef22446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:10.810421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823411db-d95b-4209-a6b6-f755cef22446 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:10.810749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823411db-d95b-4209-a6b6-f755cef22446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:10.811128 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823411db-d95b-4209-a6b6-f755cef22446 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:10.812045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823411db-d95b-4209-a6b6-f755cef22446 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:24:10.812364 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823411db-d95b-4209-a6b6-f755cef22446 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:10.814207 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823411db-d95b-4209-a6b6-f755cef22446 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:10.829506 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1211/2412] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:24:10.833701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11e798-3894-47e7-aaaa-824838ac72f7 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:10.843519 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11e798-3894-47e7-aaaa-824838ac72f7 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:10.849479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a23e65-5cf7-4f4f-84f6-a287f7e35a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:10.850146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a23e65-5cf7-4f4f-84f6-a287f7e35a78 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:10.850461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a23e65-5cf7-4f4f-84f6-a287f7e35a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:10.850743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a23e65-5cf7-4f4f-84f6-a287f7e35a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:10.852289 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a23e65-5cf7-4f4f-84f6-a287f7e35a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:10.854009 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a23e65-5cf7-4f4f-84f6-a287f7e35a78 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:10.959858 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1202/2413] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:24:10 2022] POST /identity/v3/projects => generated 369 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:24:10.961133 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823411db-d95b-4209-a6b6-f755cef22446 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:10.972699 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823411db-d95b-4209-a6b6-f755cef22446 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:10.996290 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:11.049964 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a5ba0-4527-4a12-8dd4-13d8ce53372e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:11.050852 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a5ba0-4527-4a12-8dd4-13d8ce53372e admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:11.051250 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a5ba0-4527-4a12-8dd4-13d8ce53372e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:11.051633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a5ba0-4527-4a12-8dd4-13d8ce53372e admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:11.052767 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a5ba0-4527-4a12-8dd4-13d8ce53372e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:11.054983 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a5ba0-4527-4a12-8dd4-13d8ce53372e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:11.155946 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1212/2414] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:24:10 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:24:11.160247 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a23e65-5cf7-4f4f-84f6-a287f7e35a78 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:11.171456 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a23e65-5cf7-4f4f-84f6-a287f7e35a78 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:11.176827 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2010fd-ec88-42b9-94b6-32a6d8fcb3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:11.177602 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2010fd-ec88-42b9-94b6-32a6d8fcb3ec admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:11.177991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2010fd-ec88-42b9-94b6-32a6d8fcb3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:11.178404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2010fd-ec88-42b9-94b6-32a6d8fcb3ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:11.179427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2010fd-ec88-42b9-94b6-32a6d8fcb3ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:11.181790 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2010fd-ec88-42b9-94b6-32a6d8fcb3ec admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:11.194049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1203/2415] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:24:11.198216 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a5ba0-4527-4a12-8dd4-13d8ce53372e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:11.208945 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a5ba0-4527-4a12-8dd4-13d8ce53372e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:11.213761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e7775-c115-43f3-a66e-ba9f72cbbd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:11.214467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e7775-c115-43f3-a66e-ba9f72cbbd14 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:11.214762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e7775-c115-43f3-a66e-ba9f72cbbd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:11.215042 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e7775-c115-43f3-a66e-ba9f72cbbd14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:11.215776 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e7775-c115-43f3-a66e-ba9f72cbbd14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:11.217388 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e7775-c115-43f3-a66e-ba9f72cbbd14 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:11.228388 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1213/2416] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:24:11.232572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2010fd-ec88-42b9-94b6-32a6d8fcb3ec admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:11.242786 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2010fd-ec88-42b9-94b6-32a6d8fcb3ec admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:11.247334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e700a7-aa9c-4dfb-a9a8-67b502d639f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:11.248228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e700a7-aa9c-4dfb-a9a8-67b502d639f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:11.248512 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e700a7-aa9c-4dfb-a9a8-67b502d639f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:11.248804 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e700a7-aa9c-4dfb-a9a8-67b502d639f7 admin admin] PATH_INFO: `/v3/projects/28a38335875d4c98a10fb2b5592c9d56/users/9f87250cea094c8da2dae79e1064634c/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:11.264629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e700a7-aa9c-4dfb-a9a8-67b502d639f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28a38335875d4c98a10fb2b5592c9d56, user_id=9f87250cea094c8da2dae79e1064634c, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:11.268311 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e700a7-aa9c-4dfb-a9a8-67b502d639f7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:11.392565 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1204/2417] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:24:11 2022] PUT /identity/v3/projects/28a38335875d4c98a10fb2b5592c9d56/users/9f87250cea094c8da2dae79e1064634c/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:24:11.402978 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8741c98-f955-49e1-b845-9df464e81ebf None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:11.403467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8741c98-f955-49e1-b845-9df464e81ebf None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:11.403811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8741c98-f955-49e1-b845-9df464e81ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:11.432637 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e8741c98-f955-49e1-b845-9df464e81ebf None None] MFA Rules not processed for user `9f87250cea094c8da2dae79e1064634c`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:11.434417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e8741c98-f955-49e1-b845-9df464e81ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:11.520791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1214/2418] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:11 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:11.533673 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e700a7-aa9c-4dfb-a9a8-67b502d639f7 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:11.535558 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b6e700a7-aa9c-4dfb-a9a8-67b502d639f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:11.596002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e700a7-aa9c-4dfb-a9a8-67b502d639f7 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:11.599677 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd94dd-e5b2-402c-a21c-84da17e1b01f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61459e15d327405b88a2fdfc1fc0fecd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:11.600329 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd94dd-e5b2-402c-a21c-84da17e1b01f service glance] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:11.600628 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd94dd-e5b2-402c-a21c-84da17e1b01f service glance] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:11.600909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd94dd-e5b2-402c-a21c-84da17e1b01f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:11.610726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd94dd-e5b2-402c-a21c-84da17e1b01f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:11.612111 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd94dd-e5b2-402c-a21c-84da17e1b01f service glance] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:11.621646 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1205/2419] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:24:11 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:24:21.517264 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=56,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:21.617129 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:29.959359 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8741c98-f955-49e1-b845-9df464e81ebf None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:29.961968 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e8741c98-f955-49e1-b845-9df464e81ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:30.039522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8741c98-f955-49e1-b845-9df464e81ebf None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:30.043530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e3727c-b2ce-4e15-a88d-2ceea9df2028 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:30.044189 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3727c-b2ce-4e15-a88d-2ceea9df2028 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:30.044467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3727c-b2ce-4e15-a88d-2ceea9df2028 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:30.044864 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3727c-b2ce-4e15-a88d-2ceea9df2028 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:30.046974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-37e3727c-b2ce-4e15-a88d-2ceea9df2028 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:30.122601 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e3727c-b2ce-4e15-a88d-2ceea9df2028 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:30.124045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e3727c-b2ce-4e15-a88d-2ceea9df2028 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:30.134770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1215/2420] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:24:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:24:33.655289 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd94dd-e5b2-402c-a21c-84da17e1b01f service glance] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:33.656867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-72dd94dd-e5b2-402c-a21c-84da17e1b01f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:33.747381 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd94dd-e5b2-402c-a21c-84da17e1b01f service glance] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:33.753412 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968230cf-09de-462e-818b-6299d2469bfc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7662e61619d44c92af3ffacbacba3913', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:33.754080 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968230cf-09de-462e-818b-6299d2469bfc service placement] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:33.754363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968230cf-09de-462e-818b-6299d2469bfc service placement] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:33.754675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968230cf-09de-462e-818b-6299d2469bfc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:33.756506 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-968230cf-09de-462e-818b-6299d2469bfc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:33.770063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968230cf-09de-462e-818b-6299d2469bfc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:33.771723 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968230cf-09de-462e-818b-6299d2469bfc service placement] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:33.784202 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1206/2421] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:24:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:24:34.109092 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e3727c-b2ce-4e15-a88d-2ceea9df2028 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:34.110725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-37e3727c-b2ce-4e15-a88d-2ceea9df2028 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:34.123625 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e3727c-b2ce-4e15-a88d-2ceea9df2028 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:34.130155 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14f6d1-6674-4be0-9923-1c0e4b48ea61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:34.131080 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14f6d1-6674-4be0-9923-1c0e4b48ea61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:34.131366 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14f6d1-6674-4be0-9923-1c0e4b48ea61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:34.131646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14f6d1-6674-4be0-9923-1c0e4b48ea61 admin admin] PATH_INFO: `/v3/users/114d227e9645409e9082cf9b09b05c3b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:34.132928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14f6d1-6674-4be0-9923-1c0e4b48ea61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=114d227e9645409e9082cf9b09b05c3b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:34.134964 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14f6d1-6674-4be0-9923-1c0e4b48ea61 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:34.334619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7c14f6d1-6674-4be0-9923-1c0e4b48ea61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '114d227e9645409e9082cf9b09b05c3b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:34.346806 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7c14f6d1-6674-4be0-9923-1c0e4b48ea61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '114d227e9645409e9082cf9b09b05c3b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:34.347476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7c14f6d1-6674-4be0-9923-1c0e4b48ea61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '114d227e9645409e9082cf9b09b05c3b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:34.348076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7c14f6d1-6674-4be0-9923-1c0e4b48ea61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '114d227e9645409e9082cf9b09b05c3b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:34.354820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7c14f6d1-6674-4be0-9923-1c0e4b48ea61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '114d227e9645409e9082cf9b09b05c3b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:34.422667 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1216/2422] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:24:34 2022] DELETE /identity/v3/users/114d227e9645409e9082cf9b09b05c3b => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:34.658896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968230cf-09de-462e-818b-6299d2469bfc service placement] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:34.669741 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968230cf-09de-462e-818b-6299d2469bfc service placement] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:34.732747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6b68c-77e4-4c8c-a391-cbfb55570027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:34.733610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6b68c-77e4-4c8c-a391-cbfb55570027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:34.733903 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6b68c-77e4-4c8c-a391-cbfb55570027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:34.734165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6b68c-77e4-4c8c-a391-cbfb55570027 admin admin] PATH_INFO: `/v3/projects/ab7130ecbc4d487db66baebeab8550b0` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:34.735386 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6b68c-77e4-4c8c-a391-cbfb55570027 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab7130ecbc4d487db66baebeab8550b0)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:34.737910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6b68c-77e4-4c8c-a391-cbfb55570027 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:34.853158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e5a6b68c-77e4-4c8c-a391-cbfb55570027 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab7130ecbc4d487db66baebeab8550b0', 'request_id': 'req-e5a6b68c-77e4-4c8c-a391-cbfb55570027'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:34.937329 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e5a6b68c-77e4-4c8c-a391-cbfb55570027 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab7130ecbc4d487db66baebeab8550b0', 'request_id': 'req-e5a6b68c-77e4-4c8c-a391-cbfb55570027'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:34.942888 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e5a6b68c-77e4-4c8c-a391-cbfb55570027 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab7130ecbc4d487db66baebeab8550b0', 'request_id': 'req-e5a6b68c-77e4-4c8c-a391-cbfb55570027'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:34.955756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e5a6b68c-77e4-4c8c-a391-cbfb55570027 admin admin] The token cache is being invalidate because project ab7130ecbc4d487db66baebeab8550b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:24:34.956759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e5a6b68c-77e4-4c8c-a391-cbfb55570027 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:34.957446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-e5a6b68c-77e4-4c8c-a391-cbfb55570027 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:34.959057 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1207/2423] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:24:34 2022] DELETE /identity/v3/projects/ab7130ecbc4d487db66baebeab8550b0 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:34.967706 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f1491-f01c-43ef-b4b4-574c0ac770c1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:34.967982 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f1491-f01c-43ef-b4b4-574c0ac770c1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:34.968265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f1491-f01c-43ef-b4b4-574c0ac770c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:34.987817 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-933f1491-f01c-43ef-b4b4-574c0ac770c1 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:34.989046 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-933f1491-f01c-43ef-b4b4-574c0ac770c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:35.006654 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:35.050575 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1217/2424] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:35.058274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c944a-929d-4b48-b97a-e52a6426f69f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:35.058624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c944a-929d-4b48-b97a-e52a6426f69f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:35.058906 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133c944a-929d-4b48-b97a-e52a6426f69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:35.077288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-133c944a-929d-4b48-b97a-e52a6426f69f None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:35.078520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-133c944a-929d-4b48-b97a-e52a6426f69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:35.087106 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1208/2425] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:35.091238 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f1491-f01c-43ef-b4b4-574c0ac770c1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:35.101956 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f1491-f01c-43ef-b4b4-574c0ac770c1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:35.108197 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fbc90-e046-45b8-8ac2-408c20f9012b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:35.109337 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fbc90-e046-45b8-8ac2-408c20f9012b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:35.110062 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fbc90-e046-45b8-8ac2-408c20f9012b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:35.110619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fbc90-e046-45b8-8ac2-408c20f9012b admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:35.112025 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69fbc90-e046-45b8-8ac2-408c20f9012b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:24:35.112572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69fbc90-e046-45b8-8ac2-408c20f9012b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:35.115297 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69fbc90-e046-45b8-8ac2-408c20f9012b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:35.134250 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1218/2426] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:24:35.138864 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c944a-929d-4b48-b97a-e52a6426f69f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:35.149238 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133c944a-929d-4b48-b97a-e52a6426f69f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:35.154117 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2654a9a-9b2c-4764-9e37-ee553dae997c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:35.154792 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2654a9a-9b2c-4764-9e37-ee553dae997c admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:35.155072 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2654a9a-9b2c-4764-9e37-ee553dae997c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:35.155346 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2654a9a-9b2c-4764-9e37-ee553dae997c admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:35.156031 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2654a9a-9b2c-4764-9e37-ee553dae997c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:35.157969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2654a9a-9b2c-4764-9e37-ee553dae997c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:35.167996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:35.236849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1209/2427] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:24:35 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:24:35.240762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fbc90-e046-45b8-8ac2-408c20f9012b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:35.251051 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fbc90-e046-45b8-8ac2-408c20f9012b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:35.307479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889401db-5e60-4564-929f-31eec75b3d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:35.308140 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889401db-5e60-4564-929f-31eec75b3d5d admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:35.308428 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889401db-5e60-4564-929f-31eec75b3d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:35.308707 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889401db-5e60-4564-929f-31eec75b3d5d admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:35.309391 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889401db-5e60-4564-929f-31eec75b3d5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:35.311352 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889401db-5e60-4564-929f-31eec75b3d5d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:35.389515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1219/2428] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:24:35 2022] POST /identity/v3/users => generated 282 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:24:35.393626 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2654a9a-9b2c-4764-9e37-ee553dae997c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:35.405113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2654a9a-9b2c-4764-9e37-ee553dae997c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:35.409873 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d382ec-2d66-4611-94f9-6544efbfe47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:35.410662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d382ec-2d66-4611-94f9-6544efbfe47d admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:35.410953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d382ec-2d66-4611-94f9-6544efbfe47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:35.411234 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d382ec-2d66-4611-94f9-6544efbfe47d admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:35.411984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d382ec-2d66-4611-94f9-6544efbfe47d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:35.413618 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d382ec-2d66-4611-94f9-6544efbfe47d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:35.426944 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1210/2429] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:24:35.431059 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889401db-5e60-4564-929f-31eec75b3d5d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:35.441808 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889401db-5e60-4564-929f-31eec75b3d5d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:35.446533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c138e6-ea08-4c0d-96fd-a298ece6bf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:35.447195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c138e6-ea08-4c0d-96fd-a298ece6bf42 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:35.447484 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c138e6-ea08-4c0d-96fd-a298ece6bf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:35.447765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c138e6-ea08-4c0d-96fd-a298ece6bf42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:35.448502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c138e6-ea08-4c0d-96fd-a298ece6bf42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:35.450105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c138e6-ea08-4c0d-96fd-a298ece6bf42 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:35.460213 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1220/2430] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:24:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:24:35.464535 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d382ec-2d66-4611-94f9-6544efbfe47d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:35.475255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d382ec-2d66-4611-94f9-6544efbfe47d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:35.479791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454381a-5fb6-4427-b0af-60f05e701418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:35.480693 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454381a-5fb6-4427-b0af-60f05e701418 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:35.480973 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454381a-5fb6-4427-b0af-60f05e701418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:35.481265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454381a-5fb6-4427-b0af-60f05e701418 admin admin] PATH_INFO: `/v3/projects/f0840a08d6fb43ee9370d6a0d106d8dc/users/a9a3f458404a4223a0d023dcbffaa6d1/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:35.496610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454381a-5fb6-4427-b0af-60f05e701418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0840a08d6fb43ee9370d6a0d106d8dc, user_id=a9a3f458404a4223a0d023dcbffaa6d1, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:35.499240 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454381a-5fb6-4427-b0af-60f05e701418 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:35.567952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1211/2431] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:24:35 2022] PUT /identity/v3/projects/f0840a08d6fb43ee9370d6a0d106d8dc/users/a9a3f458404a4223a0d023dcbffaa6d1/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:35.578257 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6778e0-2c7b-4d54-84fe-1b4c72494b0e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:35.578587 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6778e0-2c7b-4d54-84fe-1b4c72494b0e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:35.578874 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6778e0-2c7b-4d54-84fe-1b4c72494b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:35.605678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3e6778e0-2c7b-4d54-84fe-1b4c72494b0e None None] MFA Rules not processed for user `a9a3f458404a4223a0d023dcbffaa6d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:35.606918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3e6778e0-2c7b-4d54-84fe-1b4c72494b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:35.669770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1221/2432] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:35 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:24:35.674199 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454381a-5fb6-4427-b0af-60f05e701418 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:35.740975 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454381a-5fb6-4427-b0af-60f05e701418 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:35.745683 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14fb0c-52bf-46d1-a623-3f5c5df023ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:35.746915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14fb0c-52bf-46d1-a623-3f5c5df023ae admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:35.746915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14fb0c-52bf-46d1-a623-3f5c5df023ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:35.746915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14fb0c-52bf-46d1-a623-3f5c5df023ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:35.747628 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14fb0c-52bf-46d1-a623-3f5c5df023ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:35.749588 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14fb0c-52bf-46d1-a623-3f5c5df023ae admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:35.837476 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1212/2433] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:24:35 2022] POST /identity/v3/projects => generated 365 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:24:35.841654 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6778e0-2c7b-4d54-84fe-1b4c72494b0e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:35.852905 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6778e0-2c7b-4d54-84fe-1b4c72494b0e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:35.865819 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:35.908528 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e8adc-bea8-470b-b1d6-26f0ccc86351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:35.909221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e8adc-bea8-470b-b1d6-26f0ccc86351 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:35.909508 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e8adc-bea8-470b-b1d6-26f0ccc86351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:35.910342 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e8adc-bea8-470b-b1d6-26f0ccc86351 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:35.911078 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e8adc-bea8-470b-b1d6-26f0ccc86351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:35.913092 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e8adc-bea8-470b-b1d6-26f0ccc86351 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:36.041405 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1222/2434] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:24:35 2022] POST /identity/v3/users => generated 289 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:24:36.044715 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14fb0c-52bf-46d1-a623-3f5c5df023ae admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:36.055957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14fb0c-52bf-46d1-a623-3f5c5df023ae admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:36.060533 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6d207-2525-4628-916d-713c4f83d91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:36.061192 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6d207-2525-4628-916d-713c4f83d91a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:36.061475 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6d207-2525-4628-916d-713c4f83d91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:36.061754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f6d207-2525-4628-916d-713c4f83d91a admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:36.062570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f6d207-2525-4628-916d-713c4f83d91a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:36.064164 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f6d207-2525-4628-916d-713c4f83d91a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:36.075568 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1213/2435] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:24:36.079893 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e8adc-bea8-470b-b1d6-26f0ccc86351 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:36.090774 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e8adc-bea8-470b-b1d6-26f0ccc86351 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:36.095372 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07b359-a8a4-43ed-a158-323f5b004d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:36.096251 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07b359-a8a4-43ed-a158-323f5b004d8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:36.096535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07b359-a8a4-43ed-a158-323f5b004d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:36.096813 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07b359-a8a4-43ed-a158-323f5b004d8c admin admin] PATH_INFO: `/v3/projects/ce6db806708e40fa9be062c39f4f76ea/users/a4af69bb220f4a688f7bb25cf78b43d0/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:36.112107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07b359-a8a4-43ed-a158-323f5b004d8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce6db806708e40fa9be062c39f4f76ea, user_id=a4af69bb220f4a688f7bb25cf78b43d0, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:36.115426 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07b359-a8a4-43ed-a158-323f5b004d8c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:36.182597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1223/2436] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:24:36 2022] PUT /identity/v3/projects/ce6db806708e40fa9be062c39f4f76ea/users/a4af69bb220f4a688f7bb25cf78b43d0/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:36.186603 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6d207-2525-4628-916d-713c4f83d91a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:36.196802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f6d207-2525-4628-916d-713c4f83d91a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:36.253851 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a01a5bb-5936-4ee9-8eda-ca3d4f033e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:36.254511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01a5bb-5936-4ee9-8eda-ca3d4f033e9f admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:36.254807 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01a5bb-5936-4ee9-8eda-ca3d4f033e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:36.255097 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01a5bb-5936-4ee9-8eda-ca3d4f033e9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:36.255820 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a01a5bb-5936-4ee9-8eda-ca3d4f033e9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:36.257418 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a01a5bb-5936-4ee9-8eda-ca3d4f033e9f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:36.267327 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1214/2437] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:24:36.272048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07b359-a8a4-43ed-a158-323f5b004d8c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:36.282542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07b359-a8a4-43ed-a158-323f5b004d8c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:36.287590 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0e1bb-9d91-4b3a-ab07-39a4c9f0699a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:36.288472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0e1bb-9d91-4b3a-ab07-39a4c9f0699a admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:36.288761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0e1bb-9d91-4b3a-ab07-39a4c9f0699a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:36.289038 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0e1bb-9d91-4b3a-ab07-39a4c9f0699a admin admin] PATH_INFO: `/v3/projects/ce6db806708e40fa9be062c39f4f76ea/users/a4af69bb220f4a688f7bb25cf78b43d0/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:36.291066 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0e1bb-9d91-4b3a-ab07-39a4c9f0699a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce6db806708e40fa9be062c39f4f76ea, user_id=a4af69bb220f4a688f7bb25cf78b43d0, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:36.293781 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0e1bb-9d91-4b3a-ab07-39a4c9f0699a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:36.304819 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1224/2438] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:24:36 2022] PUT /identity/v3/projects/ce6db806708e40fa9be062c39f4f76ea/users/a4af69bb220f4a688f7bb25cf78b43d0/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:36.316464 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd630d-f888-41de-b584-25c14b25099d None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:36.316764 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd630d-f888-41de-b584-25c14b25099d None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:36.317045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd630d-f888-41de-b584-25c14b25099d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:36.343116 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-b6bd630d-f888-41de-b584-25c14b25099d None None] MFA Rules not processed for user `a4af69bb220f4a688f7bb25cf78b43d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:36.344321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b6bd630d-f888-41de-b584-25c14b25099d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:36.356893 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:36.418390 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1215/2439] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:36 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:36.430620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0e1bb-9d91-4b3a-ab07-39a4c9f0699a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:36.432318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-40f0e1bb-9d91-4b3a-ab07-39a4c9f0699a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:36.490859 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0e1bb-9d91-4b3a-ab07-39a4c9f0699a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:36.494711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602730f-51a7-4315-a3f9-c4b3d9080145 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:36.495355 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602730f-51a7-4315-a3f9-c4b3d9080145 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:36.495645 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602730f-51a7-4315-a3f9-c4b3d9080145 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:36.495928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602730f-51a7-4315-a3f9-c4b3d9080145 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:36.505365 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602730f-51a7-4315-a3f9-c4b3d9080145 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:36.506853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602730f-51a7-4315-a3f9-c4b3d9080145 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:36.516419 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1225/2440] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:24:36 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:24:36.678461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd630d-f888-41de-b584-25c14b25099d None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:36.688067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd630d-f888-41de-b584-25c14b25099d None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:36.692253 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa18bf2-3de8-4aaf-88c6-e510ef5503f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:36.692898 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa18bf2-3de8-4aaf-88c6-e510ef5503f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:36.693185 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa18bf2-3de8-4aaf-88c6-e510ef5503f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:36.693464 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa18bf2-3de8-4aaf-88c6-e510ef5503f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:36.751607 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa18bf2-3de8-4aaf-88c6-e510ef5503f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:36.753036 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa18bf2-3de8-4aaf-88c6-e510ef5503f7 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:36.763192 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1216/2441] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:24:36 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:24:38.312544 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602730f-51a7-4315-a3f9-c4b3d9080145 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:38.314071 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-1602730f-51a7-4315-a3f9-c4b3d9080145 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:38.374180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602730f-51a7-4315-a3f9-c4b3d9080145 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:38.378493 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b2fe9-a958-44d4-98ea-2eebd644a001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:38.379139 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b2fe9-a958-44d4-98ea-2eebd644a001 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:38.379424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b2fe9-a958-44d4-98ea-2eebd644a001 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:38.379703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b2fe9-a958-44d4-98ea-2eebd644a001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:38.381958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-203b2fe9-a958-44d4-98ea-2eebd644a001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:38.411470 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:38.447546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b2fe9-a958-44d4-98ea-2eebd644a001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:38.449026 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b2fe9-a958-44d4-98ea-2eebd644a001 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:38.458781 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1226/2442] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:24:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:24:42.860140 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa18bf2-3de8-4aaf-88c6-e510ef5503f7 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:42.861634 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5fa18bf2-3de8-4aaf-88c6-e510ef5503f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:42.877438 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa18bf2-3de8-4aaf-88c6-e510ef5503f7 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:42.882259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945f578-5f94-42fd-83d2-d7abc295365d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:42.883179 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945f578-5f94-42fd-83d2-d7abc295365d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:42.883466 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945f578-5f94-42fd-83d2-d7abc295365d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:42.883745 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945f578-5f94-42fd-83d2-d7abc295365d admin admin] PATH_INFO: `/v3/users/066d1521db464b4996f6daf6548dcc38` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:42.884943 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945f578-5f94-42fd-83d2-d7abc295365d admin admin] RBAC: Authorizing `identity:delete_user(user_id=066d1521db464b4996f6daf6548dcc38)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:42.887060 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945f578-5f94-42fd-83d2-d7abc295365d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:43.115275 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8945f578-5f94-42fd-83d2-d7abc295365d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '066d1521db464b4996f6daf6548dcc38'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:43.128176 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8945f578-5f94-42fd-83d2-d7abc295365d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '066d1521db464b4996f6daf6548dcc38'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:43.195675 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8945f578-5f94-42fd-83d2-d7abc295365d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '066d1521db464b4996f6daf6548dcc38'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:43.196381 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8945f578-5f94-42fd-83d2-d7abc295365d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '066d1521db464b4996f6daf6548dcc38'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:43.197007 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8945f578-5f94-42fd-83d2-d7abc295365d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '066d1521db464b4996f6daf6548dcc38'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:43.210756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1217/2443] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:24:42 2022] DELETE /identity/v3/users/066d1521db464b4996f6daf6548dcc38 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:43.449125 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b2fe9-a958-44d4-98ea-2eebd644a001 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:43.461486 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b2fe9-a958-44d4-98ea-2eebd644a001 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:43.538159 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70358bd0-ff7f-4def-8dcf-5d9514939f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:43.539193 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70358bd0-ff7f-4def-8dcf-5d9514939f8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:43.539761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70358bd0-ff7f-4def-8dcf-5d9514939f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:43.540120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70358bd0-ff7f-4def-8dcf-5d9514939f8c admin admin] PATH_INFO: `/v3/projects/ba92bd9c1afc47008582aba7ec0c5266` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:43.542430 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70358bd0-ff7f-4def-8dcf-5d9514939f8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba92bd9c1afc47008582aba7ec0c5266)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:43.545856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70358bd0-ff7f-4def-8dcf-5d9514939f8c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:43.675460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70358bd0-ff7f-4def-8dcf-5d9514939f8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba92bd9c1afc47008582aba7ec0c5266', 'request_id': 'req-70358bd0-ff7f-4def-8dcf-5d9514939f8c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:43.682505 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70358bd0-ff7f-4def-8dcf-5d9514939f8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba92bd9c1afc47008582aba7ec0c5266', 'request_id': 'req-70358bd0-ff7f-4def-8dcf-5d9514939f8c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:43.688028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70358bd0-ff7f-4def-8dcf-5d9514939f8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba92bd9c1afc47008582aba7ec0c5266', 'request_id': 'req-70358bd0-ff7f-4def-8dcf-5d9514939f8c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:43.795265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70358bd0-ff7f-4def-8dcf-5d9514939f8c admin admin] The token cache is being invalidate because project ba92bd9c1afc47008582aba7ec0c5266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:24:43.795560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70358bd0-ff7f-4def-8dcf-5d9514939f8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:43.796181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-70358bd0-ff7f-4def-8dcf-5d9514939f8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:43.797739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1227/2444] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:24:43 2022] DELETE /identity/v3/projects/ba92bd9c1afc47008582aba7ec0c5266 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:24:43.803515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdedd2-8ade-4bf7-a316-4d0c7b7db958 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:43.803838 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdedd2-8ade-4bf7-a316-4d0c7b7db958 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:43.804146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdedd2-8ade-4bf7-a316-4d0c7b7db958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:43.822347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-30cdedd2-8ade-4bf7-a316-4d0c7b7db958 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:43.823547 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-30cdedd2-8ade-4bf7-a316-4d0c7b7db958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:43.840333 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:43.883934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1218/2445] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:43.891601 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc7286-1d1f-4e26-ba4a-1de36ff91e9b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:43.891896 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc7286-1d1f-4e26-ba4a-1de36ff91e9b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:43.892179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc7286-1d1f-4e26-ba4a-1de36ff91e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:43.910776 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d8bc7286-1d1f-4e26-ba4a-1de36ff91e9b None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:43.912524 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d8bc7286-1d1f-4e26-ba4a-1de36ff91e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:43.920688 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1228/2446] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:43.924924 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdedd2-8ade-4bf7-a316-4d0c7b7db958 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:43.935128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdedd2-8ade-4bf7-a316-4d0c7b7db958 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:43.939676 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89bbc6b-3082-4192-b9b9-e1811e0e1145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:43.940327 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89bbc6b-3082-4192-b9b9-e1811e0e1145 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:43.940614 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89bbc6b-3082-4192-b9b9-e1811e0e1145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:43.940892 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89bbc6b-3082-4192-b9b9-e1811e0e1145 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:43.941646 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89bbc6b-3082-4192-b9b9-e1811e0e1145 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:24:43.941956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89bbc6b-3082-4192-b9b9-e1811e0e1145 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:43.944290 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89bbc6b-3082-4192-b9b9-e1811e0e1145 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:43.956052 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1219/2447] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:24:43.960362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc7286-1d1f-4e26-ba4a-1de36ff91e9b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:43.969771 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc7286-1d1f-4e26-ba4a-1de36ff91e9b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:43.974728 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318cc316-d619-4786-b650-0b753cd92fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:43.975373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318cc316-d619-4786-b650-0b753cd92fac admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:43.975658 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318cc316-d619-4786-b650-0b753cd92fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:43.975933 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318cc316-d619-4786-b650-0b753cd92fac admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:43.976669 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318cc316-d619-4786-b650-0b753cd92fac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:43.978630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318cc316-d619-4786-b650-0b753cd92fac admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:44.048889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1229/2448] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:24:43 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:24:44.053473 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89bbc6b-3082-4192-b9b9-e1811e0e1145 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:44.063631 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89bbc6b-3082-4192-b9b9-e1811e0e1145 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:44.120184 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93795db1-91a4-49d9-8a3a-d76bfa2c0493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:44.120854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93795db1-91a4-49d9-8a3a-d76bfa2c0493 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:44.121141 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93795db1-91a4-49d9-8a3a-d76bfa2c0493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:44.121424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93795db1-91a4-49d9-8a3a-d76bfa2c0493 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:44.122131 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93795db1-91a4-49d9-8a3a-d76bfa2c0493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:44.124098 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93795db1-91a4-49d9-8a3a-d76bfa2c0493 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:44.246419 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1220/2449] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:24:44 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:24:44.250558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318cc316-d619-4786-b650-0b753cd92fac admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:44.260954 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318cc316-d619-4786-b650-0b753cd92fac admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:44.265472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209d353-d5fd-45ec-8157-3263cb887f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:44.266149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209d353-d5fd-45ec-8157-3263cb887f6b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:44.266504 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209d353-d5fd-45ec-8157-3263cb887f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:44.266788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209d353-d5fd-45ec-8157-3263cb887f6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:44.267549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209d353-d5fd-45ec-8157-3263cb887f6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:44.269229 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209d353-d5fd-45ec-8157-3263cb887f6b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:44.280347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1230/2450] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:24:44.284279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93795db1-91a4-49d9-8a3a-d76bfa2c0493 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:44.295066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93795db1-91a4-49d9-8a3a-d76bfa2c0493 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:44.299680 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26830941-8233-4aab-b04a-0cb33597953c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:44.300334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26830941-8233-4aab-b04a-0cb33597953c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:44.300622 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26830941-8233-4aab-b04a-0cb33597953c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:44.300902 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26830941-8233-4aab-b04a-0cb33597953c admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:44.301653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26830941-8233-4aab-b04a-0cb33597953c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:44.304117 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26830941-8233-4aab-b04a-0cb33597953c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:44.313933 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1221/2451] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:24:44.318097 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209d353-d5fd-45ec-8157-3263cb887f6b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:44.327860 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209d353-d5fd-45ec-8157-3263cb887f6b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:44.332294 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05adc5-658c-49ee-a80a-5c53f8277a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:44.333178 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05adc5-658c-49ee-a80a-5c53f8277a4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:44.333466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05adc5-658c-49ee-a80a-5c53f8277a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:44.333798 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05adc5-658c-49ee-a80a-5c53f8277a4d admin admin] PATH_INFO: `/v3/projects/31e28df63efa487a8bb4d3000f3c6e81/users/6a3e633eaf2b4a44bbc653d8f61e9e2d/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:44.344581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:44.348403 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05adc5-658c-49ee-a80a-5c53f8277a4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e28df63efa487a8bb4d3000f3c6e81, user_id=6a3e633eaf2b4a44bbc653d8f61e9e2d, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:44.351851 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05adc5-658c-49ee-a80a-5c53f8277a4d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:44.445307 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1231/2452] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:24:44 2022] PUT /identity/v3/projects/31e28df63efa487a8bb4d3000f3c6e81/users/6a3e633eaf2b4a44bbc653d8f61e9e2d/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:44.456317 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa8bb5-a45d-4079-8f15-01dac04849b6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:44.456614 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa8bb5-a45d-4079-8f15-01dac04849b6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:44.456930 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa8bb5-a45d-4079-8f15-01dac04849b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:44.484060 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ebfa8bb5-a45d-4079-8f15-01dac04849b6 None None] MFA Rules not processed for user `6a3e633eaf2b4a44bbc653d8f61e9e2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:44.486280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ebfa8bb5-a45d-4079-8f15-01dac04849b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:44.549459 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1222/2453] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:44.554003 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05adc5-658c-49ee-a80a-5c53f8277a4d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:44.620594 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05adc5-658c-49ee-a80a-5c53f8277a4d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:44.629986 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9de239-9569-43c3-a6c3-d3e3eba16ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:44.629986 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9de239-9569-43c3-a6c3-d3e3eba16ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:44.629986 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9de239-9569-43c3-a6c3-d3e3eba16ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:44.629986 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9de239-9569-43c3-a6c3-d3e3eba16ed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:44.630291 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9de239-9569-43c3-a6c3-d3e3eba16ed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:44.630291 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9de239-9569-43c3-a6c3-d3e3eba16ed5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:44.761491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1232/2454] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:24:44 2022] POST /identity/v3/projects => generated 377 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:24:44.765515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa8bb5-a45d-4079-8f15-01dac04849b6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:44.777001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa8bb5-a45d-4079-8f15-01dac04849b6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:44.789742 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:44.836891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8b989-ad94-40b4-9d39-26cb41455bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:44.837564 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8b989-ad94-40b4-9d39-26cb41455bb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:44.837880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8b989-ad94-40b4-9d39-26cb41455bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:44.838158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8b989-ad94-40b4-9d39-26cb41455bb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:44.838890 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8b989-ad94-40b4-9d39-26cb41455bb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:44.840806 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8b989-ad94-40b4-9d39-26cb41455bb9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:44.934435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1223/2455] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:24:44 2022] POST /identity/v3/users => generated 295 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:24:44.937792 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9de239-9569-43c3-a6c3-d3e3eba16ed5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:44.952783 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9de239-9569-43c3-a6c3-d3e3eba16ed5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:44.957779 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf5e59-e77f-43f7-b2f2-069275624a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:44.959102 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf5e59-e77f-43f7-b2f2-069275624a27 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:44.959616 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf5e59-e77f-43f7-b2f2-069275624a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:44.960172 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf5e59-e77f-43f7-b2f2-069275624a27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:44.960912 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf5e59-e77f-43f7-b2f2-069275624a27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:44.962562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf5e59-e77f-43f7-b2f2-069275624a27 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:44.973491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1233/2456] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:24:44.978323 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8b989-ad94-40b4-9d39-26cb41455bb9 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:44.988260 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8b989-ad94-40b4-9d39-26cb41455bb9 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:44.993029 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ecfc9-2594-4aa8-a680-abbd5e699447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:44.993923 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ecfc9-2594-4aa8-a680-abbd5e699447 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:44.994208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ecfc9-2594-4aa8-a680-abbd5e699447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:44.994520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ecfc9-2594-4aa8-a680-abbd5e699447 admin admin] PATH_INFO: `/v3/projects/197df0cb183f40c681f824473b8112ee/users/be588ea03fff4eccb172e2bbaa20ffb5/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:45.009915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ecfc9-2594-4aa8-a680-abbd5e699447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=197df0cb183f40c681f824473b8112ee, user_id=be588ea03fff4eccb172e2bbaa20ffb5, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:45.012686 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ecfc9-2594-4aa8-a680-abbd5e699447 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:45.081435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1224/2457] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:24:44 2022] PUT /identity/v3/projects/197df0cb183f40c681f824473b8112ee/users/be588ea03fff4eccb172e2bbaa20ffb5/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:45.086585 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf5e59-e77f-43f7-b2f2-069275624a27 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:45.096745 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf5e59-e77f-43f7-b2f2-069275624a27 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:45.156645 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd2130-884b-4141-9dd1-5dd622abf5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:45.157367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd2130-884b-4141-9dd1-5dd622abf5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:45.157728 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd2130-884b-4141-9dd1-5dd622abf5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:45.158009 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd2130-884b-4141-9dd1-5dd622abf5f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:45.158767 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd2130-884b-4141-9dd1-5dd622abf5f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:45.160370 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd2130-884b-4141-9dd1-5dd622abf5f5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:45.171555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1234/2458] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:24:45.175655 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ecfc9-2594-4aa8-a680-abbd5e699447 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:45.186595 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ecfc9-2594-4aa8-a680-abbd5e699447 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:45.191173 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bfee1-235d-451b-9044-1e8006bdc829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:45.192052 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bfee1-235d-451b-9044-1e8006bdc829 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:45.192337 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bfee1-235d-451b-9044-1e8006bdc829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:45.193359 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bfee1-235d-451b-9044-1e8006bdc829 admin admin] PATH_INFO: `/v3/projects/197df0cb183f40c681f824473b8112ee/users/be588ea03fff4eccb172e2bbaa20ffb5/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:45.195464 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bfee1-235d-451b-9044-1e8006bdc829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=197df0cb183f40c681f824473b8112ee, user_id=be588ea03fff4eccb172e2bbaa20ffb5, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:45.198138 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bfee1-235d-451b-9044-1e8006bdc829 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:45.208280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1225/2459] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:24:45 2022] PUT /identity/v3/projects/197df0cb183f40c681f824473b8112ee/users/be588ea03fff4eccb172e2bbaa20ffb5/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:45.218524 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a880f-fe96-465d-9eb6-cbc5111f9384 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:45.218818 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a880f-fe96-465d-9eb6-cbc5111f9384 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:45.219128 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a880f-fe96-465d-9eb6-cbc5111f9384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:45.246253 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-398a880f-fe96-465d-9eb6-cbc5111f9384 None None] MFA Rules not processed for user `be588ea03fff4eccb172e2bbaa20ffb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:45.247483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-398a880f-fe96-465d-9eb6-cbc5111f9384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:45.293737 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:45.314920 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1235/2460] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:45 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:45.326916 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bfee1-235d-451b-9044-1e8006bdc829 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:45.328564 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-322bfee1-235d-451b-9044-1e8006bdc829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:45.386337 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bfee1-235d-451b-9044-1e8006bdc829 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:45.390264 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d17e2a-e253-4cb8-a236-d11ec2f9e856 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:45.390991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d17e2a-e253-4cb8-a236-d11ec2f9e856 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:45.391276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d17e2a-e253-4cb8-a236-d11ec2f9e856 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:45.391629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d17e2a-e253-4cb8-a236-d11ec2f9e856 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:45.449058 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d17e2a-e253-4cb8-a236-d11ec2f9e856 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:45.450523 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d17e2a-e253-4cb8-a236-d11ec2f9e856 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:45.459368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1226/2461] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:24:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:24:45.637607 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a880f-fe96-465d-9eb6-cbc5111f9384 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:45.639202 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-398a880f-fe96-465d-9eb6-cbc5111f9384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:45.706019 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a880f-fe96-465d-9eb6-cbc5111f9384 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:45.710764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e3708-9579-4125-b62a-ceea769ed24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:45.711643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e3708-9579-4125-b62a-ceea769ed24e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:45.711922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e3708-9579-4125-b62a-ceea769ed24e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:45.712196 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e3708-9579-4125-b62a-ceea769ed24e admin admin] PATH_INFO: `/v3/users/9f87250cea094c8da2dae79e1064634c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:45.713385 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e3708-9579-4125-b62a-ceea769ed24e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f87250cea094c8da2dae79e1064634c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:45.715341 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5e3708-9579-4125-b62a-ceea769ed24e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:45.947928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6e5e3708-9579-4125-b62a-ceea769ed24e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f87250cea094c8da2dae79e1064634c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:45.974748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6e5e3708-9579-4125-b62a-ceea769ed24e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f87250cea094c8da2dae79e1064634c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:45.975026 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6e5e3708-9579-4125-b62a-ceea769ed24e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f87250cea094c8da2dae79e1064634c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:45.975765 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6e5e3708-9579-4125-b62a-ceea769ed24e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f87250cea094c8da2dae79e1064634c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:45.985547 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6e5e3708-9579-4125-b62a-ceea769ed24e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f87250cea094c8da2dae79e1064634c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:46.080580 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1236/2462] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:24:45 2022] DELETE /identity/v3/users/9f87250cea094c8da2dae79e1064634c => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:46.085462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d1409-f6f8-4074-9e3a-9c1f0aeb7086 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:46.085925 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d1409-f6f8-4074-9e3a-9c1f0aeb7086 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:46.086031 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d1409-f6f8-4074-9e3a-9c1f0aeb7086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:46.105218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f88d1409-f6f8-4074-9e3a-9c1f0aeb7086 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:46.106617 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f88d1409-f6f8-4074-9e3a-9c1f0aeb7086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:46.128572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:46.171663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1227/2463] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:46.181744 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e3708-9579-4125-b62a-ceea769ed24e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:46.183425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6e5e3708-9579-4125-b62a-ceea769ed24e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:46.257730 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e3708-9579-4125-b62a-ceea769ed24e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:46.262248 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da724e94-62a4-4aeb-ac47-a96dc7d7b4b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:46.263355 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da724e94-62a4-4aeb-ac47-a96dc7d7b4b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:46.263802 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da724e94-62a4-4aeb-ac47-a96dc7d7b4b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:46.264386 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da724e94-62a4-4aeb-ac47-a96dc7d7b4b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:46.275162 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da724e94-62a4-4aeb-ac47-a96dc7d7b4b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:46.276754 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da724e94-62a4-4aeb-ac47-a96dc7d7b4b0 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:46.287217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1237/2464] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:24:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:24:46.798882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d1409-f6f8-4074-9e3a-9c1f0aeb7086 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:46.800424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f88d1409-f6f8-4074-9e3a-9c1f0aeb7086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:46.819171 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d1409-f6f8-4074-9e3a-9c1f0aeb7086 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:46.830714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae953c-25a2-400d-8f0c-22ad64bd6d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:46.830714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae953c-25a2-400d-8f0c-22ad64bd6d9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:46.830714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae953c-25a2-400d-8f0c-22ad64bd6d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:46.830714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae953c-25a2-400d-8f0c-22ad64bd6d9d admin admin] PATH_INFO: `/v3/projects/28a38335875d4c98a10fb2b5592c9d56` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:46.832000 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae953c-25a2-400d-8f0c-22ad64bd6d9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=28a38335875d4c98a10fb2b5592c9d56)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:46.834288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae953c-25a2-400d-8f0c-22ad64bd6d9d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:47.008482 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c9ae953c-25a2-400d-8f0c-22ad64bd6d9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28a38335875d4c98a10fb2b5592c9d56', 'request_id': 'req-c9ae953c-25a2-400d-8f0c-22ad64bd6d9d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:47.092997 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c9ae953c-25a2-400d-8f0c-22ad64bd6d9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28a38335875d4c98a10fb2b5592c9d56', 'request_id': 'req-c9ae953c-25a2-400d-8f0c-22ad64bd6d9d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:47.100476 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c9ae953c-25a2-400d-8f0c-22ad64bd6d9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28a38335875d4c98a10fb2b5592c9d56', 'request_id': 'req-c9ae953c-25a2-400d-8f0c-22ad64bd6d9d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:47.111703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c9ae953c-25a2-400d-8f0c-22ad64bd6d9d admin admin] The token cache is being invalidate because project 28a38335875d4c98a10fb2b5592c9d56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:24:47.111995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c9ae953c-25a2-400d-8f0c-22ad64bd6d9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:47.112690 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c9ae953c-25a2-400d-8f0c-22ad64bd6d9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:47.114335 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1228/2465] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:24:46 2022] DELETE /identity/v3/projects/28a38335875d4c98a10fb2b5592c9d56 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:47.119395 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd7c63-5426-4019-9fbe-0d37fba0d028 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:47.119889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd7c63-5426-4019-9fbe-0d37fba0d028 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:47.120033 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd7c63-5426-4019-9fbe-0d37fba0d028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:47.133771 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:47.139675 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-51fd7c63-5426-4019-9fbe-0d37fba0d028 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:47.140903 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-51fd7c63-5426-4019-9fbe-0d37fba0d028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:47.202962 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1238/2466] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:47.213562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a848d4-937a-4240-8d21-3d36bbdfcd95 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:47.213562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a848d4-937a-4240-8d21-3d36bbdfcd95 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:47.213562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a848d4-937a-4240-8d21-3d36bbdfcd95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:47.231799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-84a848d4-937a-4240-8d21-3d36bbdfcd95 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:47.232993 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-84a848d4-937a-4240-8d21-3d36bbdfcd95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:47.244978 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1229/2467] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:47.246038 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd7c63-5426-4019-9fbe-0d37fba0d028 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:47.256729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd7c63-5426-4019-9fbe-0d37fba0d028 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:47.261484 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa70150-e811-4b5b-85f3-6cbb17fe640e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:47.269326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa70150-e811-4b5b-85f3-6cbb17fe640e admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:47.269326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa70150-e811-4b5b-85f3-6cbb17fe640e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:47.269326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa70150-e811-4b5b-85f3-6cbb17fe640e admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:47.269326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa70150-e811-4b5b-85f3-6cbb17fe640e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:24:47.269326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa70150-e811-4b5b-85f3-6cbb17fe640e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:47.269326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa70150-e811-4b5b-85f3-6cbb17fe640e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:47.279200 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1239/2468] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:24:47.283267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a848d4-937a-4240-8d21-3d36bbdfcd95 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:47.295223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a848d4-937a-4240-8d21-3d36bbdfcd95 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:47.299981 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067c9b9-8dc8-4a80-b26b-dd1d878cfcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:47.301059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067c9b9-8dc8-4a80-b26b-dd1d878cfcc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:47.301541 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067c9b9-8dc8-4a80-b26b-dd1d878cfcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:47.301799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067c9b9-8dc8-4a80-b26b-dd1d878cfcc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:47.302545 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067c9b9-8dc8-4a80-b26b-dd1d878cfcc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:47.305340 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067c9b9-8dc8-4a80-b26b-dd1d878cfcc2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:47.393078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1230/2469] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:24:47 2022] POST /identity/v3/projects => generated 389 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:24:47.398084 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa70150-e811-4b5b-85f3-6cbb17fe640e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:47.410050 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa70150-e811-4b5b-85f3-6cbb17fe640e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:47.489783 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d2d86-ea33-4371-b960-90f37bf4b2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:47.489783 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d2d86-ea33-4371-b960-90f37bf4b2f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:47.489783 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d2d86-ea33-4371-b960-90f37bf4b2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:47.489783 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576d2d86-ea33-4371-b960-90f37bf4b2f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:47.491728 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d2d86-ea33-4371-b960-90f37bf4b2f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:47.492686 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576d2d86-ea33-4371-b960-90f37bf4b2f0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:47.541555 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067c9b9-8dc8-4a80-b26b-dd1d878cfcc2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:47.545882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a067c9b9-8dc8-4a80-b26b-dd1d878cfcc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:47.603149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1240/2470] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:24:47 2022] POST /identity/v3/users => generated 295 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:24:47.603375 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067c9b9-8dc8-4a80-b26b-dd1d878cfcc2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:47.604113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b4f6c-04b4-4ded-8d8a-4e223b944ac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:47.604422 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d2d86-ea33-4371-b960-90f37bf4b2f0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:47.604773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b4f6c-04b4-4ded-8d8a-4e223b944ac7 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:47.605059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b4f6c-04b4-4ded-8d8a-4e223b944ac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:47.605342 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b4f6c-04b4-4ded-8d8a-4e223b944ac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:47.607154 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-083b4f6c-04b4-4ded-8d8a-4e223b944ac7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:47.614169 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576d2d86-ea33-4371-b960-90f37bf4b2f0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:47.620034 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126fad5e-6a08-4907-8828-027034134796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:47.620692 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fad5e-6a08-4907-8828-027034134796 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:47.620982 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fad5e-6a08-4907-8828-027034134796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:47.621261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fad5e-6a08-4907-8828-027034134796 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:47.621985 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126fad5e-6a08-4907-8828-027034134796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:47.623259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b4f6c-04b4-4ded-8d8a-4e223b944ac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:47.623629 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126fad5e-6a08-4907-8828-027034134796 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:47.624709 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083b4f6c-04b4-4ded-8d8a-4e223b944ac7 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:47.636202 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1241/2471] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:24:47.640127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126fad5e-6a08-4907-8828-027034134796 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:47.641407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1231/2472] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:24:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:24:47.653896 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126fad5e-6a08-4907-8828-027034134796 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:47.660377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509b60e-02f1-485b-a889-dc4c4c2f3c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:47.661597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509b60e-02f1-485b-a889-dc4c4c2f3c97 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:47.662044 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509b60e-02f1-485b-a889-dc4c4c2f3c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:47.662457 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509b60e-02f1-485b-a889-dc4c4c2f3c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:47.663425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509b60e-02f1-485b-a889-dc4c4c2f3c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:47.665327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509b60e-02f1-485b-a889-dc4c4c2f3c97 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:47.677756 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1242/2473] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:24:47.684068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b4f6c-04b4-4ded-8d8a-4e223b944ac7 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:47.695013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b4f6c-04b4-4ded-8d8a-4e223b944ac7 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:47.721608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b147b-9c47-4645-b58d-964f7b982641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:47.721608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b147b-9c47-4645-b58d-964f7b982641 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:47.721608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b147b-9c47-4645-b58d-964f7b982641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:47.721608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b147b-9c47-4645-b58d-964f7b982641 admin admin] PATH_INFO: `/v3/projects/eebba33fe87a4f32ba4f697eadf56bb8/users/c51a2c1727f745e2a6068597015a7a06/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:47.736120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b147b-9c47-4645-b58d-964f7b982641 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eebba33fe87a4f32ba4f697eadf56bb8, user_id=c51a2c1727f745e2a6068597015a7a06, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:47.740455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b147b-9c47-4645-b58d-964f7b982641 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:47.819141 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:47.825316 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1232/2474] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:24:47 2022] PUT /identity/v3/projects/eebba33fe87a4f32ba4f697eadf56bb8/users/c51a2c1727f745e2a6068597015a7a06/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:47.838456 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ed56f-2497-4603-86e5-5e7e1fc31161 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:47.839049 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ed56f-2497-4603-86e5-5e7e1fc31161 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:47.839390 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ed56f-2497-4603-86e5-5e7e1fc31161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:47.868153 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-b59ed56f-2497-4603-86e5-5e7e1fc31161 None None] MFA Rules not processed for user `c51a2c1727f745e2a6068597015a7a06`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:47.869363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b59ed56f-2497-4603-86e5-5e7e1fc31161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:47.921881 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:47.940420 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1243/2475] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:47.951363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b147b-9c47-4645-b58d-964f7b982641 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:47.953158 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-de6b147b-9c47-4645-b58d-964f7b982641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:48.012122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b147b-9c47-4645-b58d-964f7b982641 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:48.016124 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42e167-b8ca-4c77-94a7-55cab7d04cc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:48.019506 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42e167-b8ca-4c77-94a7-55cab7d04cc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:48.019506 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42e167-b8ca-4c77-94a7-55cab7d04cc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:48.019506 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d42e167-b8ca-4c77-94a7-55cab7d04cc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:48.027776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d42e167-b8ca-4c77-94a7-55cab7d04cc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:48.029219 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d42e167-b8ca-4c77-94a7-55cab7d04cc4 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:48.046533 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1233/2476] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:24:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:24:51.402633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ed56f-2497-4603-86e5-5e7e1fc31161 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:51.404497 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b59ed56f-2497-4603-86e5-5e7e1fc31161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:51.476862 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ed56f-2497-4603-86e5-5e7e1fc31161 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:51.481510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613bc20-d983-4336-9c33-69eda2ed35aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:51.482572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613bc20-d983-4336-9c33-69eda2ed35aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:51.482853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613bc20-d983-4336-9c33-69eda2ed35aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:51.483136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613bc20-d983-4336-9c33-69eda2ed35aa admin admin] PATH_INFO: `/v3/users/9ffa029a3b1e4919b00e58a9737816d2` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:51.484335 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a613bc20-d983-4336-9c33-69eda2ed35aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ffa029a3b1e4919b00e58a9737816d2)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:51.486330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a613bc20-d983-4336-9c33-69eda2ed35aa admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:51.675296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a613bc20-d983-4336-9c33-69eda2ed35aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ffa029a3b1e4919b00e58a9737816d2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:51.687327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a613bc20-d983-4336-9c33-69eda2ed35aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ffa029a3b1e4919b00e58a9737816d2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:51.688002 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a613bc20-d983-4336-9c33-69eda2ed35aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ffa029a3b1e4919b00e58a9737816d2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:51.688607 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a613bc20-d983-4336-9c33-69eda2ed35aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ffa029a3b1e4919b00e58a9737816d2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:51.695965 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a613bc20-d983-4336-9c33-69eda2ed35aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ffa029a3b1e4919b00e58a9737816d2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:51.777863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1244/2477] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:24:51 2022] DELETE /identity/v3/users/9ffa029a3b1e4919b00e58a9737816d2 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:51.910525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42e167-b8ca-4c77-94a7-55cab7d04cc4 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:51.912131 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5d42e167-b8ca-4c77-94a7-55cab7d04cc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:51.923458 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d42e167-b8ca-4c77-94a7-55cab7d04cc4 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:51.974622 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00246338-5d07-444c-a24f-fd3327959d34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:51.975272 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00246338-5d07-444c-a24f-fd3327959d34 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:51.975593 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00246338-5d07-444c-a24f-fd3327959d34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:51.975870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00246338-5d07-444c-a24f-fd3327959d34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:51.977750 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-00246338-5d07-444c-a24f-fd3327959d34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:52.040849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00246338-5d07-444c-a24f-fd3327959d34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:52.042308 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00246338-5d07-444c-a24f-fd3327959d34 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:52.052667 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1234/2478] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:24:51 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:24:52.061116 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613bc20-d983-4336-9c33-69eda2ed35aa admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:52.061116 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a613bc20-d983-4336-9c33-69eda2ed35aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:52.139285 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613bc20-d983-4336-9c33-69eda2ed35aa admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:52.145695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07145446-9943-48e8-9514-bea7db135686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:52.151922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07145446-9943-48e8-9514-bea7db135686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:52.151922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07145446-9943-48e8-9514-bea7db135686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:52.151922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07145446-9943-48e8-9514-bea7db135686 admin admin] PATH_INFO: `/v3/projects/d56eee171a5c4f728bec99eef1f3b0a3` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:52.151922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07145446-9943-48e8-9514-bea7db135686 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d56eee171a5c4f728bec99eef1f3b0a3)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:52.151922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07145446-9943-48e8-9514-bea7db135686 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:52.163267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:52.276345 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-07145446-9943-48e8-9514-bea7db135686 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd56eee171a5c4f728bec99eef1f3b0a3', 'request_id': 'req-07145446-9943-48e8-9514-bea7db135686'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:52.286782 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-07145446-9943-48e8-9514-bea7db135686 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd56eee171a5c4f728bec99eef1f3b0a3', 'request_id': 'req-07145446-9943-48e8-9514-bea7db135686'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:52.294096 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-07145446-9943-48e8-9514-bea7db135686 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd56eee171a5c4f728bec99eef1f3b0a3', 'request_id': 'req-07145446-9943-48e8-9514-bea7db135686'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:52.415415 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-07145446-9943-48e8-9514-bea7db135686 admin admin] The token cache is being invalidate because project d56eee171a5c4f728bec99eef1f3b0a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:24:52.415987 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-07145446-9943-48e8-9514-bea7db135686 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:52.416571 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-07145446-9943-48e8-9514-bea7db135686 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:52.418579 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1245/2479] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:24:52 2022] DELETE /identity/v3/projects/d56eee171a5c4f728bec99eef1f3b0a3 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:52.423615 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a345a-3100-486b-959f-eb4c8ad26b45 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:52.423936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a345a-3100-486b-959f-eb4c8ad26b45 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:52.424237 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a345a-3100-486b-959f-eb4c8ad26b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:52.445635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-cd8a345a-3100-486b-959f-eb4c8ad26b45 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:52.446874 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-cd8a345a-3100-486b-959f-eb4c8ad26b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:52.505925 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:52.520225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1235/2480] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:52.528749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2fc7e-a5b5-4a86-a67a-bc3fff8ac2b4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:52.529047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2fc7e-a5b5-4a86-a67a-bc3fff8ac2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:52.529334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2fc7e-a5b5-4a86-a67a-bc3fff8ac2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:52.549129 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7da2fc7e-a5b5-4a86-a67a-bc3fff8ac2b4 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:52.550397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7da2fc7e-a5b5-4a86-a67a-bc3fff8ac2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:52.564463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1246/2481] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:52.569002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a345a-3100-486b-959f-eb4c8ad26b45 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:52.578896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a345a-3100-486b-959f-eb4c8ad26b45 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:52.583713 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78384393-9489-4f07-96b2-62be3c7e0920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:52.584636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78384393-9489-4f07-96b2-62be3c7e0920 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:52.584919 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78384393-9489-4f07-96b2-62be3c7e0920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:52.585197 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78384393-9489-4f07-96b2-62be3c7e0920 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:52.585921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78384393-9489-4f07-96b2-62be3c7e0920 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:24:52.586229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78384393-9489-4f07-96b2-62be3c7e0920 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:52.587867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78384393-9489-4f07-96b2-62be3c7e0920 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:52.603769 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1236/2482] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:24:52.604940 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2fc7e-a5b5-4a86-a67a-bc3fff8ac2b4 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:52.606190 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78384393-9489-4f07-96b2-62be3c7e0920 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:52.608090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-78384393-9489-4f07-96b2-62be3c7e0920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:52.614804 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2fc7e-a5b5-4a86-a67a-bc3fff8ac2b4 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:52.619530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398bf6b-f113-41a3-ba7b-373781bb7906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:52.620177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398bf6b-f113-41a3-ba7b-373781bb7906 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:52.626314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398bf6b-f113-41a3-ba7b-373781bb7906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:52.626314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398bf6b-f113-41a3-ba7b-373781bb7906 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:52.626314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398bf6b-f113-41a3-ba7b-373781bb7906 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:52.626658 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78384393-9489-4f07-96b2-62be3c7e0920 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:52.626757 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398bf6b-f113-41a3-ba7b-373781bb7906 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:52.626854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0ff624-77ea-4acb-9fb8-56c0775874db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:52.627459 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ff624-77ea-4acb-9fb8-56c0775874db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:52.627740 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ff624-77ea-4acb-9fb8-56c0775874db admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:52.628026 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0ff624-77ea-4acb-9fb8-56c0775874db admin admin] PATH_INFO: `/v3/users/a9a3f458404a4223a0d023dcbffaa6d1` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:52.629258 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ff624-77ea-4acb-9fb8-56c0775874db admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9a3f458404a4223a0d023dcbffaa6d1)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:52.631227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0ff624-77ea-4acb-9fb8-56c0775874db admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:52.727246 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1247/2483] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:24:52 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:24:52.731229 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398bf6b-f113-41a3-ba7b-373781bb7906 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:52.791756 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398bf6b-f113-41a3-ba7b-373781bb7906 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:52.796335 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c26b2-1c4d-4d9f-94d1-d342dcdbb48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:52.796999 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c26b2-1c4d-4d9f-94d1-d342dcdbb48a admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:52.797282 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c26b2-1c4d-4d9f-94d1-d342dcdbb48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:52.797555 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c26b2-1c4d-4d9f-94d1-d342dcdbb48a admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:52.798242 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c26b2-1c4d-4d9f-94d1-d342dcdbb48a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:52.800200 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c26b2-1c4d-4d9f-94d1-d342dcdbb48a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:52.901397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0b0ff624-77ea-4acb-9fb8-56c0775874db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9a3f458404a4223a0d023dcbffaa6d1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:52.913800 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0b0ff624-77ea-4acb-9fb8-56c0775874db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9a3f458404a4223a0d023dcbffaa6d1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:52.935922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1248/2484] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:24:52 2022] POST /identity/v3/users => generated 287 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:24:52.939151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c26b2-1c4d-4d9f-94d1-d342dcdbb48a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:52.949824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c26b2-1c4d-4d9f-94d1-d342dcdbb48a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:52.955489 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a33b09-acb8-4ed9-9668-f5dea647d9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:52.956354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a33b09-acb8-4ed9-9668-f5dea647d9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:52.956454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a33b09-acb8-4ed9-9668-f5dea647d9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:52.956711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a33b09-acb8-4ed9-9668-f5dea647d9c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:52.957510 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a33b09-acb8-4ed9-9668-f5dea647d9c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:52.959776 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a33b09-acb8-4ed9-9668-f5dea647d9c5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:52.970619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1249/2485] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:24:52.975138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a33b09-acb8-4ed9-9668-f5dea647d9c5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:52.983890 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0b0ff624-77ea-4acb-9fb8-56c0775874db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9a3f458404a4223a0d023dcbffaa6d1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:52.984531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0b0ff624-77ea-4acb-9fb8-56c0775874db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9a3f458404a4223a0d023dcbffaa6d1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:52.984847 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a33b09-acb8-4ed9-9668-f5dea647d9c5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:52.985121 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0b0ff624-77ea-4acb-9fb8-56c0775874db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9a3f458404a4223a0d023dcbffaa6d1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:52.989697 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94b12b-e9d7-4145-95a2-db29ab128cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:52.994700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94b12b-e9d7-4145-95a2-db29ab128cdc admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:52.994700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94b12b-e9d7-4145-95a2-db29ab128cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:52.994700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94b12b-e9d7-4145-95a2-db29ab128cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:52.994700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94b12b-e9d7-4145-95a2-db29ab128cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:52.995598 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94b12b-e9d7-4145-95a2-db29ab128cdc admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:53.000717 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1237/2486] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:24:52 2022] DELETE /identity/v3/users/a9a3f458404a4223a0d023dcbffaa6d1 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:24:53.006553 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfceada-ea73-4133-a691-e1b28d37c76b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:53.006830 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfceada-ea73-4133-a691-e1b28d37c76b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:53.007113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfceada-ea73-4133-a691-e1b28d37c76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:53.010803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1250/2487] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:24:53.011741 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94b12b-e9d7-4145-95a2-db29ab128cdc admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:53.013381 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ea94b12b-e9d7-4145-95a2-db29ab128cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:53.026940 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94b12b-e9d7-4145-95a2-db29ab128cdc admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:53.029264 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2cfceada-ea73-4133-a691-e1b28d37c76b None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:53.030632 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2cfceada-ea73-4133-a691-e1b28d37c76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:53.071120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:53.091178 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18afeba-89e1-41d5-8ab7-0019f5bbec3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:53.092108 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18afeba-89e1-41d5-8ab7-0019f5bbec3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:53.092410 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18afeba-89e1-41d5-8ab7-0019f5bbec3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:53.092687 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18afeba-89e1-41d5-8ab7-0019f5bbec3b admin admin] PATH_INFO: `/v3/projects/106010db0a25438092fc802cffb34384/users/388b53a8e7c5418da6ce8b9a55b7dae0/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:53.094629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1238/2488] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:24:53.104529 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfceada-ea73-4133-a691-e1b28d37c76b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:53.105952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2cfceada-ea73-4133-a691-e1b28d37c76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:53.107863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18afeba-89e1-41d5-8ab7-0019f5bbec3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=106010db0a25438092fc802cffb34384, user_id=388b53a8e7c5418da6ce8b9a55b7dae0, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:53.110915 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18afeba-89e1-41d5-8ab7-0019f5bbec3b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:53.164401 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfceada-ea73-4133-a691-e1b28d37c76b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:53.168045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da44237-d5ca-4440-a3cb-21dc8c7ab2a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:53.168694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da44237-d5ca-4440-a3cb-21dc8c7ab2a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:53.168977 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da44237-d5ca-4440-a3cb-21dc8c7ab2a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:53.169252 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da44237-d5ca-4440-a3cb-21dc8c7ab2a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:53.173066 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1251/2489] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:24:53 2022] PUT /identity/v3/projects/106010db0a25438092fc802cffb34384/users/388b53a8e7c5418da6ce8b9a55b7dae0/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:24:53.178128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:53.182156 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da44237-d5ca-4440-a3cb-21dc8c7ab2a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:53.183499 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92380a16-442f-4ef7-9d32-1c41e487b2b3 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:53.183654 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da44237-d5ca-4440-a3cb-21dc8c7ab2a2 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:53.183835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92380a16-442f-4ef7-9d32-1c41e487b2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:53.184102 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92380a16-442f-4ef7-9d32-1c41e487b2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:53.194282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1239/2490] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:24:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:24:53.211954 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-92380a16-442f-4ef7-9d32-1c41e487b2b3 None None] MFA Rules not processed for user `388b53a8e7c5418da6ce8b9a55b7dae0`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:53.213296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-92380a16-442f-4ef7-9d32-1c41e487b2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:53.279712 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1252/2491] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:53.291225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da44237-d5ca-4440-a3cb-21dc8c7ab2a2 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:53.292630 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2da44237-d5ca-4440-a3cb-21dc8c7ab2a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:53.371061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da44237-d5ca-4440-a3cb-21dc8c7ab2a2 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:53.441287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55495f0f-453f-4b81-8f96-0dca95801ed5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:53.441958 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55495f0f-453f-4b81-8f96-0dca95801ed5 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:53.442274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55495f0f-453f-4b81-8f96-0dca95801ed5 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:53.442662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55495f0f-453f-4b81-8f96-0dca95801ed5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:53.451390 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55495f0f-453f-4b81-8f96-0dca95801ed5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:53.452891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55495f0f-453f-4b81-8f96-0dca95801ed5 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:53.463098 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1240/2492] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:24:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:24:53.667641 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92380a16-442f-4ef7-9d32-1c41e487b2b3 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:53.669227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-92380a16-442f-4ef7-9d32-1c41e487b2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:53.735767 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92380a16-442f-4ef7-9d32-1c41e487b2b3 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:53.741446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e44f4-3466-4a02-bf11-b084b370c419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:53.742308 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e44f4-3466-4a02-bf11-b084b370c419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:53.742630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e44f4-3466-4a02-bf11-b084b370c419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:53.742910 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e44f4-3466-4a02-bf11-b084b370c419 admin admin] PATH_INFO: `/v3/projects/f0840a08d6fb43ee9370d6a0d106d8dc` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:53.744091 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e44f4-3466-4a02-bf11-b084b370c419 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0840a08d6fb43ee9370d6a0d106d8dc)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:53.746163 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e44f4-3466-4a02-bf11-b084b370c419 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:53.858488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-897e44f4-3466-4a02-bf11-b084b370c419 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0840a08d6fb43ee9370d6a0d106d8dc', 'request_id': 'req-897e44f4-3466-4a02-bf11-b084b370c419'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:53.865383 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-897e44f4-3466-4a02-bf11-b084b370c419 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0840a08d6fb43ee9370d6a0d106d8dc', 'request_id': 'req-897e44f4-3466-4a02-bf11-b084b370c419'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:53.866639 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:53.871318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-897e44f4-3466-4a02-bf11-b084b370c419 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0840a08d6fb43ee9370d6a0d106d8dc', 'request_id': 'req-897e44f4-3466-4a02-bf11-b084b370c419'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:53.955140 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-897e44f4-3466-4a02-bf11-b084b370c419 admin admin] The token cache is being invalidate because project f0840a08d6fb43ee9370d6a0d106d8dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:24:53.955435 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-897e44f4-3466-4a02-bf11-b084b370c419 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:53.956067 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-897e44f4-3466-4a02-bf11-b084b370c419 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:53.957626 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1253/2493] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:24:53 2022] DELETE /identity/v3/projects/f0840a08d6fb43ee9370d6a0d106d8dc => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:53.961692 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55495f0f-453f-4b81-8f96-0dca95801ed5 service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:53.972364 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55495f0f-453f-4b81-8f96-0dca95801ed5 service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:54.029380 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42aa5d5-5fba-4959-b5da-768f8b490578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:54.030266 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42aa5d5-5fba-4959-b5da-768f8b490578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:54.030576 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42aa5d5-5fba-4959-b5da-768f8b490578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:54.030852 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42aa5d5-5fba-4959-b5da-768f8b490578 admin admin] PATH_INFO: `/v3/users/a4af69bb220f4a688f7bb25cf78b43d0` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:54.032053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42aa5d5-5fba-4959-b5da-768f8b490578 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4af69bb220f4a688f7bb25cf78b43d0)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:54.033980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42aa5d5-5fba-4959-b5da-768f8b490578 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:54.056577 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:54.231106 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b42aa5d5-5fba-4959-b5da-768f8b490578 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4af69bb220f4a688f7bb25cf78b43d0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:54.247902 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b42aa5d5-5fba-4959-b5da-768f8b490578 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4af69bb220f4a688f7bb25cf78b43d0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:54.325065 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b42aa5d5-5fba-4959-b5da-768f8b490578 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4af69bb220f4a688f7bb25cf78b43d0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:54.325700 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b42aa5d5-5fba-4959-b5da-768f8b490578 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4af69bb220f4a688f7bb25cf78b43d0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:54.326315 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-b42aa5d5-5fba-4959-b5da-768f8b490578 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4af69bb220f4a688f7bb25cf78b43d0'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:54.341051 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1241/2494] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:24:53 2022] DELETE /identity/v3/users/a4af69bb220f4a688f7bb25cf78b43d0 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:54.453604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e44f4-3466-4a02-bf11-b084b370c419 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:54.455098 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-897e44f4-3466-4a02-bf11-b084b370c419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:54.515816 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e44f4-3466-4a02-bf11-b084b370c419 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:54.574595 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cefc0d-ec75-4040-94c0-533f5867db3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:54.575274 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cefc0d-ec75-4040-94c0-533f5867db3d service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:54.575544 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cefc0d-ec75-4040-94c0-533f5867db3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:54.575825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cefc0d-ec75-4040-94c0-533f5867db3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:54.577672 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-34cefc0d-ec75-4040-94c0-533f5867db3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:54.643883 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cefc0d-ec75-4040-94c0-533f5867db3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:54.645499 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cefc0d-ec75-4040-94c0-533f5867db3d service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:54.655746 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1254/2495] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:24:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:24:54.790197 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42aa5d5-5fba-4959-b5da-768f8b490578 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:54.791687 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b42aa5d5-5fba-4959-b5da-768f8b490578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:54.877232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42aa5d5-5fba-4959-b5da-768f8b490578 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:54.882330 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295de592-35fa-4c4a-b619-9546c3fe98bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:54.883235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295de592-35fa-4c4a-b619-9546c3fe98bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:54.883525 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295de592-35fa-4c4a-b619-9546c3fe98bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:54.883845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295de592-35fa-4c4a-b619-9546c3fe98bc admin admin] PATH_INFO: `/v3/projects/ce6db806708e40fa9be062c39f4f76ea` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:54.885087 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295de592-35fa-4c4a-b619-9546c3fe98bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce6db806708e40fa9be062c39f4f76ea)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:54.887068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295de592-35fa-4c4a-b619-9546c3fe98bc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:55.000806 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-295de592-35fa-4c4a-b619-9546c3fe98bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce6db806708e40fa9be062c39f4f76ea', 'request_id': 'req-295de592-35fa-4c4a-b619-9546c3fe98bc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:55.107882 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-295de592-35fa-4c4a-b619-9546c3fe98bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce6db806708e40fa9be062c39f4f76ea', 'request_id': 'req-295de592-35fa-4c4a-b619-9546c3fe98bc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:55.115285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-295de592-35fa-4c4a-b619-9546c3fe98bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce6db806708e40fa9be062c39f4f76ea', 'request_id': 'req-295de592-35fa-4c4a-b619-9546c3fe98bc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:55.126614 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-295de592-35fa-4c4a-b619-9546c3fe98bc admin admin] The token cache is being invalidate because project ce6db806708e40fa9be062c39f4f76ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:24:55.126900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-295de592-35fa-4c4a-b619-9546c3fe98bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:55.127534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-295de592-35fa-4c4a-b619-9546c3fe98bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:24:55.129101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1242/2496] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:24:54 2022] DELETE /identity/v3/projects/ce6db806708e40fa9be062c39f4f76ea => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:24:55.134957 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bffbc4-5c51-492e-820c-51e25a86b304 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:55.135471 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bffbc4-5c51-492e-820c-51e25a86b304 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:55.135580 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bffbc4-5c51-492e-820c-51e25a86b304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:55.154960 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-47bffbc4-5c51-492e-820c-51e25a86b304 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:55.156179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-47bffbc4-5c51-492e-820c-51e25a86b304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:55.209786 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:55.227874 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1255/2497] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:55.236012 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a2d8f-39cd-4e55-8786-69bc8e1eb1fd None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:55.236309 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a2d8f-39cd-4e55-8786-69bc8e1eb1fd None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:55.236594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a2d8f-39cd-4e55-8786-69bc8e1eb1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:55.255871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-de6a2d8f-39cd-4e55-8786-69bc8e1eb1fd None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:55.257092 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-de6a2d8f-39cd-4e55-8786-69bc8e1eb1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:55.265698 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1243/2498] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:24:55.274171 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bffbc4-5c51-492e-820c-51e25a86b304 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:55.283989 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bffbc4-5c51-492e-820c-51e25a86b304 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:55.289044 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50daccfb-a06e-4ad6-8d69-070cd9b6c291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:55.289809 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50daccfb-a06e-4ad6-8d69-070cd9b6c291 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:55.290164 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50daccfb-a06e-4ad6-8d69-070cd9b6c291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:55.290546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50daccfb-a06e-4ad6-8d69-070cd9b6c291 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:55.291346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50daccfb-a06e-4ad6-8d69-070cd9b6c291 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:24:55.291748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50daccfb-a06e-4ad6-8d69-070cd9b6c291 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:55.293437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50daccfb-a06e-4ad6-8d69-070cd9b6c291 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:55.307405 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1256/2499] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:24:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:24:55.311653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a2d8f-39cd-4e55-8786-69bc8e1eb1fd None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:55.324523 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a2d8f-39cd-4e55-8786-69bc8e1eb1fd None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:55.331540 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0d9ee-05cc-4194-9a71-70282a205199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:55.332597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0d9ee-05cc-4194-9a71-70282a205199 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:55.332597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0d9ee-05cc-4194-9a71-70282a205199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:55.333389 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0d9ee-05cc-4194-9a71-70282a205199 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:55.334061 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0d9ee-05cc-4194-9a71-70282a205199 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:55.336160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0d9ee-05cc-4194-9a71-70282a205199 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:55.441587 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:55.449360 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1244/2500] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:24:55 2022] POST /identity/v3/projects => generated 367 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:24:55.453470 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50daccfb-a06e-4ad6-8d69-070cd9b6c291 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:55.463769 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50daccfb-a06e-4ad6-8d69-070cd9b6c291 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:55.522979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299737e7-6e76-474c-831d-80cb5a1d1ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:55.523651 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299737e7-6e76-474c-831d-80cb5a1d1ac7 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:55.523937 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299737e7-6e76-474c-831d-80cb5a1d1ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:55.524216 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299737e7-6e76-474c-831d-80cb5a1d1ac7 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:55.525176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299737e7-6e76-474c-831d-80cb5a1d1ac7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:55.527275 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299737e7-6e76-474c-831d-80cb5a1d1ac7 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:55.669470 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0d9ee-05cc-4194-9a71-70282a205199 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:55.675319 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1257/2501] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:24:55 2022] POST /identity/v3/users => generated 284 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:24:55.683693 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0d9ee-05cc-4194-9a71-70282a205199 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:55.688995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd89c9ab-4c3b-4077-8721-af6b5a0a202d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:55.689570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89c9ab-4c3b-4077-8721-af6b5a0a202d admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:55.689938 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89c9ab-4c3b-4077-8721-af6b5a0a202d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:55.690226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89c9ab-4c3b-4077-8721-af6b5a0a202d admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:55.691037 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd89c9ab-4c3b-4077-8721-af6b5a0a202d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:55.692971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd89c9ab-4c3b-4077-8721-af6b5a0a202d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:55.703994 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1245/2502] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:24:55.708709 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299737e7-6e76-474c-831d-80cb5a1d1ac7 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:55.719615 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299737e7-6e76-474c-831d-80cb5a1d1ac7 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:55.726734 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649e79b-57e9-4314-a44f-474366c1d001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:55.727373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649e79b-57e9-4314-a44f-474366c1d001 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:55.727666 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649e79b-57e9-4314-a44f-474366c1d001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:55.727948 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649e79b-57e9-4314-a44f-474366c1d001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:55.728928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649e79b-57e9-4314-a44f-474366c1d001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:55.730603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649e79b-57e9-4314-a44f-474366c1d001 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:55.742715 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1258/2503] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:24:55.746725 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd89c9ab-4c3b-4077-8721-af6b5a0a202d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:55.756914 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd89c9ab-4c3b-4077-8721-af6b5a0a202d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:55.762665 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29bf86-6962-4ceb-8fee-1f51120f979a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:55.763803 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29bf86-6962-4ceb-8fee-1f51120f979a admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:55.763986 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29bf86-6962-4ceb-8fee-1f51120f979a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:55.764281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29bf86-6962-4ceb-8fee-1f51120f979a admin admin] PATH_INFO: `/v3/projects/482d5d59ac014a35a72b238e18a474a2/users/0831d2b245804f3c91fc35dc75f0a75b/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:55.781434 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29bf86-6962-4ceb-8fee-1f51120f979a admin admin] RBAC: Authorizing `identity:create_grant(project_id=482d5d59ac014a35a72b238e18a474a2, user_id=0831d2b245804f3c91fc35dc75f0a75b, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:55.784244 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29bf86-6962-4ceb-8fee-1f51120f979a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:55.893114 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1246/2504] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:24:55 2022] PUT /identity/v3/projects/482d5d59ac014a35a72b238e18a474a2/users/0831d2b245804f3c91fc35dc75f0a75b/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:55.903887 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e63f6-aff4-467f-8cad-19bb78ba3f46 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:55.904186 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e63f6-aff4-467f-8cad-19bb78ba3f46 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:55.904469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e63f6-aff4-467f-8cad-19bb78ba3f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:55.934927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4f3e63f6-aff4-467f-8cad-19bb78ba3f46 None None] MFA Rules not processed for user `0831d2b245804f3c91fc35dc75f0a75b`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:55.935462 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4f3e63f6-aff4-467f-8cad-19bb78ba3f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:56.002107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1259/2505] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:55 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:56.006870 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29bf86-6962-4ceb-8fee-1f51120f979a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:56.080142 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29bf86-6962-4ceb-8fee-1f51120f979a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:56.085101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c036a-dc65-4d3b-9b9c-e35af0c606ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:56.085807 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c036a-dc65-4d3b-9b9c-e35af0c606ce admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:56.086091 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c036a-dc65-4d3b-9b9c-e35af0c606ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:56.086400 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c036a-dc65-4d3b-9b9c-e35af0c606ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:56.087272 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c036a-dc65-4d3b-9b9c-e35af0c606ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:56.089187 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3c036a-dc65-4d3b-9b9c-e35af0c606ce admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:56.195953 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1247/2506] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:24:56 2022] POST /identity/v3/projects => generated 369 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:24:56.197316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e63f6-aff4-467f-8cad-19bb78ba3f46 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:56.209284 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e63f6-aff4-467f-8cad-19bb78ba3f46 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:56.258824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:56.270851 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99698ac-c95a-4296-87f0-1b47a2ded31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:56.271634 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99698ac-c95a-4296-87f0-1b47a2ded31b admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:56.276675 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99698ac-c95a-4296-87f0-1b47a2ded31b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:56.276675 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99698ac-c95a-4296-87f0-1b47a2ded31b admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:56.276675 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99698ac-c95a-4296-87f0-1b47a2ded31b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:56.276675 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99698ac-c95a-4296-87f0-1b47a2ded31b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:56.350543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1260/2507] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:24:56 2022] POST /identity/v3/users => generated 291 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:24:56.354424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c036a-dc65-4d3b-9b9c-e35af0c606ce admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:56.367787 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3c036a-dc65-4d3b-9b9c-e35af0c606ce admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:56.371453 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81873b1b-7dea-410c-b953-aa71e318ab30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:56.372118 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81873b1b-7dea-410c-b953-aa71e318ab30 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:56.372403 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81873b1b-7dea-410c-b953-aa71e318ab30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:56.372682 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81873b1b-7dea-410c-b953-aa71e318ab30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:56.373406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81873b1b-7dea-410c-b953-aa71e318ab30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:56.375025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81873b1b-7dea-410c-b953-aa71e318ab30 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:56.393357 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1248/2508] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:24:56.394577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99698ac-c95a-4296-87f0-1b47a2ded31b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:56.408226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99698ac-c95a-4296-87f0-1b47a2ded31b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:56.430926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6a98e-75e5-40bd-88eb-addcf7b193a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:56.432356 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6a98e-75e5-40bd-88eb-addcf7b193a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:56.432356 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6a98e-75e5-40bd-88eb-addcf7b193a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:56.432562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6a98e-75e5-40bd-88eb-addcf7b193a2 admin admin] PATH_INFO: `/v3/projects/e4c1a64559cf4fc1b889f0c5efd87ca2/users/be6f5b9f5088448e84b70ed25eb5c84a/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:56.454745 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6a98e-75e5-40bd-88eb-addcf7b193a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4c1a64559cf4fc1b889f0c5efd87ca2, user_id=be6f5b9f5088448e84b70ed25eb5c84a, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:56.457620 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6a98e-75e5-40bd-88eb-addcf7b193a2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:56.578954 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1261/2509] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:24:56 2022] PUT /identity/v3/projects/e4c1a64559cf4fc1b889f0c5efd87ca2/users/be6f5b9f5088448e84b70ed25eb5c84a/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:24:56.583841 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81873b1b-7dea-410c-b953-aa71e318ab30 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:56.594428 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81873b1b-7dea-410c-b953-aa71e318ab30 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:56.658674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6955a14-ad51-4192-9bde-5f300a30e4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:56.659332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6955a14-ad51-4192-9bde-5f300a30e4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:56.659619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6955a14-ad51-4192-9bde-5f300a30e4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:56.659898 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6955a14-ad51-4192-9bde-5f300a30e4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:56.660658 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6955a14-ad51-4192-9bde-5f300a30e4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:56.662521 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6955a14-ad51-4192-9bde-5f300a30e4e4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:56.677478 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6a98e-75e5-40bd-88eb-addcf7b193a2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:56.678850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1249/2510] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:24:56.689768 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6a98e-75e5-40bd-88eb-addcf7b193a2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:56.694258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3121f6-8709-4bdb-9b27-c380f869d167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:56.695201 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3121f6-8709-4bdb-9b27-c380f869d167 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:56.695491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3121f6-8709-4bdb-9b27-c380f869d167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:56.695768 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3121f6-8709-4bdb-9b27-c380f869d167 admin admin] PATH_INFO: `/v3/projects/e4c1a64559cf4fc1b889f0c5efd87ca2/users/be6f5b9f5088448e84b70ed25eb5c84a/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:56.698277 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3121f6-8709-4bdb-9b27-c380f869d167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4c1a64559cf4fc1b889f0c5efd87ca2, user_id=be6f5b9f5088448e84b70ed25eb5c84a, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:56.701113 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3121f6-8709-4bdb-9b27-c380f869d167 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:56.713506 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1262/2511] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:24:56 2022] PUT /identity/v3/projects/e4c1a64559cf4fc1b889f0c5efd87ca2/users/be6f5b9f5088448e84b70ed25eb5c84a/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:24:56.725941 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e197a5-fd78-4605-82e9-e53f22c8503b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:56.726423 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e197a5-fd78-4605-82e9-e53f22c8503b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:56.726742 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e197a5-fd78-4605-82e9-e53f22c8503b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:56.756716 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-69e197a5-fd78-4605-82e9-e53f22c8503b None None] MFA Rules not processed for user `be6f5b9f5088448e84b70ed25eb5c84a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:24:56.757958 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-69e197a5-fd78-4605-82e9-e53f22c8503b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:56.779283 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:24:56.863358 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1250/2512] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:24:56 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:24:56.877418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3121f6-8709-4bdb-9b27-c380f869d167 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:56.881695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-bb3121f6-8709-4bdb-9b27-c380f869d167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:56.937458 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3121f6-8709-4bdb-9b27-c380f869d167 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:56.941187 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187d467-1266-432e-894c-1652a10c4dc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:56.944729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187d467-1266-432e-894c-1652a10c4dc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:56.944729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187d467-1266-432e-894c-1652a10c4dc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:56.944729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187d467-1266-432e-894c-1652a10c4dc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:56.951803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187d467-1266-432e-894c-1652a10c4dc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:56.953237 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187d467-1266-432e-894c-1652a10c4dc6 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:56.962742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1263/2513] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:24:56 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:24:57.056568 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e197a5-fd78-4605-82e9-e53f22c8503b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:24:57.058248 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-69e197a5-fd78-4605-82e9-e53f22c8503b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:24:57.070616 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e197a5-fd78-4605-82e9-e53f22c8503b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:24:57.074560 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab8a6a-3ec0-4adb-9c8d-4955182de948 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:24:57.075216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab8a6a-3ec0-4adb-9c8d-4955182de948 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:24:57.075505 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab8a6a-3ec0-4adb-9c8d-4955182de948 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:24:57.075791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab8a6a-3ec0-4adb-9c8d-4955182de948 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:24:57.134333 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab8a6a-3ec0-4adb-9c8d-4955182de948 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:24:57.135807 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab8a6a-3ec0-4adb-9c8d-4955182de948 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:24:57.144677 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1251/2514] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:24:57 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:01.926179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187d467-1266-432e-894c-1652a10c4dc6 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:02.002489 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187d467-1266-432e-894c-1652a10c4dc6 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:02.008336 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65393278-a345-4bef-94e8-e6d410e4ac7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:02.009243 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65393278-a345-4bef-94e8-e6d410e4ac7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:02.009525 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65393278-a345-4bef-94e8-e6d410e4ac7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:02.009803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65393278-a345-4bef-94e8-e6d410e4ac7d admin admin] PATH_INFO: `/v3/users/0831d2b245804f3c91fc35dc75f0a75b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:02.011019 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65393278-a345-4bef-94e8-e6d410e4ac7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0831d2b245804f3c91fc35dc75f0a75b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:02.012955 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65393278-a345-4bef-94e8-e6d410e4ac7d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:02.225418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:02.227106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-65393278-a345-4bef-94e8-e6d410e4ac7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0831d2b245804f3c91fc35dc75f0a75b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:02.239439 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-65393278-a345-4bef-94e8-e6d410e4ac7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0831d2b245804f3c91fc35dc75f0a75b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:02.241289 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-65393278-a345-4bef-94e8-e6d410e4ac7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0831d2b245804f3c91fc35dc75f0a75b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:02.241955 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-65393278-a345-4bef-94e8-e6d410e4ac7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0831d2b245804f3c91fc35dc75f0a75b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:02.249075 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-65393278-a345-4bef-94e8-e6d410e4ac7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0831d2b245804f3c91fc35dc75f0a75b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:02.371119 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1264/2515] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:01 2022] DELETE /identity/v3/users/0831d2b245804f3c91fc35dc75f0a75b => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:02.375733 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cc08c-ca48-4116-be2d-2ad6dfe72a8c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:02.376230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cc08c-ca48-4116-be2d-2ad6dfe72a8c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:02.376325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48cc08c-ca48-4116-be2d-2ad6dfe72a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:02.394363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a48cc08c-ca48-4116-be2d-2ad6dfe72a8c None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:02.395772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a48cc08c-ca48-4116-be2d-2ad6dfe72a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:02.466193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1252/2516] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:02.476768 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65393278-a345-4bef-94e8-e6d410e4ac7d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:02.478357 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-65393278-a345-4bef-94e8-e6d410e4ac7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:02.543133 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65393278-a345-4bef-94e8-e6d410e4ac7d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:02.547323 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34665dc8-d371-4de7-a87f-a15c19db73b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:02.548252 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34665dc8-d371-4de7-a87f-a15c19db73b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:02.548612 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34665dc8-d371-4de7-a87f-a15c19db73b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:02.549581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34665dc8-d371-4de7-a87f-a15c19db73b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:02.559668 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34665dc8-d371-4de7-a87f-a15c19db73b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:02.561120 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34665dc8-d371-4de7-a87f-a15c19db73b3 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:02.570978 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1265/2517] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:03.131246 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48cc08c-ca48-4116-be2d-2ad6dfe72a8c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:03.132754 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a48cc08c-ca48-4116-be2d-2ad6dfe72a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:03.147828 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48cc08c-ca48-4116-be2d-2ad6dfe72a8c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:03.164982 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f6a34-2883-407c-af34-ccd29b1ac6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:03.166011 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f6a34-2883-407c-af34-ccd29b1ac6a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:03.166361 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f6a34-2883-407c-af34-ccd29b1ac6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:03.166717 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45f6a34-2883-407c-af34-ccd29b1ac6a9 admin admin] PATH_INFO: `/v3/projects/482d5d59ac014a35a72b238e18a474a2` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:03.168490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f6a34-2883-407c-af34-ccd29b1ac6a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=482d5d59ac014a35a72b238e18a474a2)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:03.185638 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45f6a34-2883-407c-af34-ccd29b1ac6a9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:03.320898 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c45f6a34-2883-407c-af34-ccd29b1ac6a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '482d5d59ac014a35a72b238e18a474a2', 'request_id': 'req-c45f6a34-2883-407c-af34-ccd29b1ac6a9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:03.324321 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:03.445847 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c45f6a34-2883-407c-af34-ccd29b1ac6a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '482d5d59ac014a35a72b238e18a474a2', 'request_id': 'req-c45f6a34-2883-407c-af34-ccd29b1ac6a9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:03.455459 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c45f6a34-2883-407c-af34-ccd29b1ac6a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '482d5d59ac014a35a72b238e18a474a2', 'request_id': 'req-c45f6a34-2883-407c-af34-ccd29b1ac6a9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:03.469393 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c45f6a34-2883-407c-af34-ccd29b1ac6a9 admin admin] The token cache is being invalidate because project 482d5d59ac014a35a72b238e18a474a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:03.469688 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c45f6a34-2883-407c-af34-ccd29b1ac6a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:03.470358 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c45f6a34-2883-407c-af34-ccd29b1ac6a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:03.472046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1253/2518] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:03 2022] DELETE /identity/v3/projects/482d5d59ac014a35a72b238e18a474a2 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:03.476675 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34665dc8-d371-4de7-a87f-a15c19db73b3 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:03.487362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34665dc8-d371-4de7-a87f-a15c19db73b3 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:03.554507 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0f268-0a62-474c-bb46-87d0ce68393b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:03.555338 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0f268-0a62-474c-bb46-87d0ce68393b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:03.555626 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0f268-0a62-474c-bb46-87d0ce68393b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:03.555908 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0f268-0a62-474c-bb46-87d0ce68393b admin admin] PATH_INFO: `/v3/users/be6f5b9f5088448e84b70ed25eb5c84a` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:03.557117 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0f268-0a62-474c-bb46-87d0ce68393b admin admin] RBAC: Authorizing `identity:delete_user(user_id=be6f5b9f5088448e84b70ed25eb5c84a)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:03.559121 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0f268-0a62-474c-bb46-87d0ce68393b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:03.764726 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-80a0f268-0a62-474c-bb46-87d0ce68393b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be6f5b9f5088448e84b70ed25eb5c84a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:03.782441 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-80a0f268-0a62-474c-bb46-87d0ce68393b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be6f5b9f5088448e84b70ed25eb5c84a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:03.783217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-80a0f268-0a62-474c-bb46-87d0ce68393b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be6f5b9f5088448e84b70ed25eb5c84a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:03.783835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-80a0f268-0a62-474c-bb46-87d0ce68393b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be6f5b9f5088448e84b70ed25eb5c84a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:03.785876 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:03.791447 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-80a0f268-0a62-474c-bb46-87d0ce68393b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be6f5b9f5088448e84b70ed25eb5c84a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:03.863283 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1266/2519] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:03 2022] DELETE /identity/v3/users/be6f5b9f5088448e84b70ed25eb5c84a => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:04.335007 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f6a34-2883-407c-af34-ccd29b1ac6a9 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:04.337226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c45f6a34-2883-407c-af34-ccd29b1ac6a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:04.363983 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45f6a34-2883-407c-af34-ccd29b1ac6a9 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:04.424491 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a0b5f-3935-454d-a7f9-2238828d430d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:04.425355 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a0b5f-3935-454d-a7f9-2238828d430d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:04.425641 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a0b5f-3935-454d-a7f9-2238828d430d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:04.425923 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a0b5f-3935-454d-a7f9-2238828d430d admin admin] PATH_INFO: `/v3/projects/e4c1a64559cf4fc1b889f0c5efd87ca2` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:04.427145 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a0b5f-3935-454d-a7f9-2238828d430d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4c1a64559cf4fc1b889f0c5efd87ca2)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:04.429219 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a0b5f-3935-454d-a7f9-2238828d430d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:04.508927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0f268-0a62-474c-bb46-87d0ce68393b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:04.508927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-80a0f268-0a62-474c-bb46-87d0ce68393b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:04.519389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0f268-0a62-474c-bb46-87d0ce68393b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:04.525404 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454267d-6ca4-4ba2-9fd1-d7b5a0d33ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:04.526422 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454267d-6ca4-4ba2-9fd1-d7b5a0d33ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:04.529169 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454267d-6ca4-4ba2-9fd1-d7b5a0d33ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:04.529169 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454267d-6ca4-4ba2-9fd1-d7b5a0d33ee9 admin admin] PATH_INFO: `/v3/users/388b53a8e7c5418da6ce8b9a55b7dae0` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:04.529169 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454267d-6ca4-4ba2-9fd1-d7b5a0d33ee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=388b53a8e7c5418da6ce8b9a55b7dae0)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:04.531659 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454267d-6ca4-4ba2-9fd1-d7b5a0d33ee9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:04.543587 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5c6a0b5f-3935-454d-a7f9-2238828d430d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4c1a64559cf4fc1b889f0c5efd87ca2', 'request_id': 'req-5c6a0b5f-3935-454d-a7f9-2238828d430d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:04.633188 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5c6a0b5f-3935-454d-a7f9-2238828d430d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4c1a64559cf4fc1b889f0c5efd87ca2', 'request_id': 'req-5c6a0b5f-3935-454d-a7f9-2238828d430d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:04.639238 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5c6a0b5f-3935-454d-a7f9-2238828d430d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4c1a64559cf4fc1b889f0c5efd87ca2', 'request_id': 'req-5c6a0b5f-3935-454d-a7f9-2238828d430d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:04.651336 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5c6a0b5f-3935-454d-a7f9-2238828d430d admin admin] The token cache is being invalidate because project e4c1a64559cf4fc1b889f0c5efd87ca2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:04.651625 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5c6a0b5f-3935-454d-a7f9-2238828d430d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:04.652251 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5c6a0b5f-3935-454d-a7f9-2238828d430d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:04.653779 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1254/2520] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:04 2022] DELETE /identity/v3/projects/e4c1a64559cf4fc1b889f0c5efd87ca2 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:04.659080 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dd827-86df-47e9-a839-f4545858f9fa None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:04.659866 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dd827-86df-47e9-a839-f4545858f9fa None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:04.659866 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56dd827-86df-47e9-a839-f4545858f9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:04.678734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e56dd827-86df-47e9-a839-f4545858f9fa None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:04.679951 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e56dd827-86df-47e9-a839-f4545858f9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:04.739021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1255/2521] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:04.746972 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448aff0d-f9a8-4b9f-b309-2a078b8d15af None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:04.747334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448aff0d-f9a8-4b9f-b309-2a078b8d15af None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:04.747623 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448aff0d-f9a8-4b9f-b309-2a078b8d15af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:04.765811 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-448aff0d-f9a8-4b9f-b309-2a078b8d15af None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:04.767044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-448aff0d-f9a8-4b9f-b309-2a078b8d15af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:04.769757 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a454267d-6ca4-4ba2-9fd1-d7b5a0d33ee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '388b53a8e7c5418da6ce8b9a55b7dae0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:04.777194 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1256/2522] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:04.781626 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448aff0d-f9a8-4b9f-b309-2a078b8d15af None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:04.782955 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a454267d-6ca4-4ba2-9fd1-d7b5a0d33ee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '388b53a8e7c5418da6ce8b9a55b7dae0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:04.783705 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a454267d-6ca4-4ba2-9fd1-d7b5a0d33ee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '388b53a8e7c5418da6ce8b9a55b7dae0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:04.784410 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a454267d-6ca4-4ba2-9fd1-d7b5a0d33ee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '388b53a8e7c5418da6ce8b9a55b7dae0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:04.786962 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:04.791238 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448aff0d-f9a8-4b9f-b309-2a078b8d15af None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:04.791993 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a454267d-6ca4-4ba2-9fd1-d7b5a0d33ee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '388b53a8e7c5418da6ce8b9a55b7dae0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:04.795940 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc8d24-57c2-47af-9321-c8bd42e06ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:04.798857 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc8d24-57c2-47af-9321-c8bd42e06ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:04.798857 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc8d24-57c2-47af-9321-c8bd42e06ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:04.798857 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc8d24-57c2-47af-9321-c8bd42e06ad4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:04.798857 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc8d24-57c2-47af-9321-c8bd42e06ad4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:25:04.798857 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc8d24-57c2-47af-9321-c8bd42e06ad4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:04.800422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc8d24-57c2-47af-9321-c8bd42e06ad4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:04.813160 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1257/2523] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:25:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:25:04.817497 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc8d24-57c2-47af-9321-c8bd42e06ad4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:04.825312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b3fc8d24-57c2-47af-9321-c8bd42e06ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:04.832370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc8d24-57c2-47af-9321-c8bd42e06ad4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:04.837089 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3253c3-77c9-42a5-b1ba-87f72fbd3cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:04.837718 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3253c3-77c9-42a5-b1ba-87f72fbd3cbb admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:04.838009 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3253c3-77c9-42a5-b1ba-87f72fbd3cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:04.838280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3253c3-77c9-42a5-b1ba-87f72fbd3cbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:04.838996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3253c3-77c9-42a5-b1ba-87f72fbd3cbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:04.840865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3253c3-77c9-42a5-b1ba-87f72fbd3cbb admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:04.879490 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1267/2524] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:04 2022] DELETE /identity/v3/users/388b53a8e7c5418da6ce8b9a55b7dae0 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:04.885203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8250b-7893-4753-8de7-e176485cd717 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:04.885443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8250b-7893-4753-8de7-e176485cd717 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:04.885731 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8250b-7893-4753-8de7-e176485cd717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:04.903722 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2ee8250b-7893-4753-8de7-e176485cd717 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:04.904925 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2ee8250b-7893-4753-8de7-e176485cd717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:04.963475 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1268/2525] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:04.972883 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8250b-7893-4753-8de7-e176485cd717 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:04.974178 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2ee8250b-7893-4753-8de7-e176485cd717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:04.999756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1258/2526] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:04 2022] POST /identity/v3/projects => generated 391 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:05.000746 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3253c3-77c9-42a5-b1ba-87f72fbd3cbb admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:05.037606 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8250b-7893-4753-8de7-e176485cd717 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:05.048650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:05.074212 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3253c3-77c9-42a5-b1ba-87f72fbd3cbb admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:05.079180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976cf5f-86aa-4ebb-9e02-ab24e3fd7a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:05.079952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976cf5f-86aa-4ebb-9e02-ab24e3fd7a4c admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:05.080335 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976cf5f-86aa-4ebb-9e02-ab24e3fd7a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:05.080714 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976cf5f-86aa-4ebb-9e02-ab24e3fd7a4c admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:05.081418 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976cf5f-86aa-4ebb-9e02-ab24e3fd7a4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:05.083394 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976cf5f-86aa-4ebb-9e02-ab24e3fd7a4c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:05.093640 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa24612-c890-4472-9409-a88f7b4aea89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:05.094299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa24612-c890-4472-9409-a88f7b4aea89 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:05.094623 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa24612-c890-4472-9409-a88f7b4aea89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:05.094940 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa24612-c890-4472-9409-a88f7b4aea89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:05.104361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa24612-c890-4472-9409-a88f7b4aea89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:05.105809 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa24612-c890-4472-9409-a88f7b4aea89 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:05.115501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1269/2527] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:25:05.180457 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1259/2528] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:04 2022] POST /identity/v3/users => generated 296 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:05.184465 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa24612-c890-4472-9409-a88f7b4aea89 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:05.194219 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa24612-c890-4472-9409-a88f7b4aea89 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:05.199767 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251f85a-2a6d-496e-b324-92169cd5fa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:05.200535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251f85a-2a6d-496e-b324-92169cd5fa67 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:05.204857 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251f85a-2a6d-496e-b324-92169cd5fa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:05.204857 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251f85a-2a6d-496e-b324-92169cd5fa67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:05.204857 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251f85a-2a6d-496e-b324-92169cd5fa67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:05.204857 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251f85a-2a6d-496e-b324-92169cd5fa67 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:05.217234 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1270/2529] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:05.218546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976cf5f-86aa-4ebb-9e02-ab24e3fd7a4c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:05.229382 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976cf5f-86aa-4ebb-9e02-ab24e3fd7a4c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:05.234012 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31972ecd-e057-40a8-8560-32799e148bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:05.234706 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31972ecd-e057-40a8-8560-32799e148bec admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:05.234996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31972ecd-e057-40a8-8560-32799e148bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:05.235280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31972ecd-e057-40a8-8560-32799e148bec admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:05.236001 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31972ecd-e057-40a8-8560-32799e148bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:05.237614 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31972ecd-e057-40a8-8560-32799e148bec admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:05.248207 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1260/2530] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:05.252778 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251f85a-2a6d-496e-b324-92169cd5fa67 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:05.263652 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251f85a-2a6d-496e-b324-92169cd5fa67 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:05.268403 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151d525-e103-41ca-b60d-ea03d6de17b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:05.269276 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151d525-e103-41ca-b60d-ea03d6de17b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:05.269562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151d525-e103-41ca-b60d-ea03d6de17b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:05.269844 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151d525-e103-41ca-b60d-ea03d6de17b0 admin admin] PATH_INFO: `/v3/projects/7b23bd7324194a7ab0438e13dcf70c4b/users/07653944f5874e02b54c9bc0869c39d1/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:05.285219 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151d525-e103-41ca-b60d-ea03d6de17b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b23bd7324194a7ab0438e13dcf70c4b, user_id=07653944f5874e02b54c9bc0869c39d1, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:05.288766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151d525-e103-41ca-b60d-ea03d6de17b0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:05.357108 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1271/2531] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:05 2022] PUT /identity/v3/projects/7b23bd7324194a7ab0438e13dcf70c4b/users/07653944f5874e02b54c9bc0869c39d1/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:05.369445 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27240d-5bc8-4843-abf6-0d46a2aedc22 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:05.369763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27240d-5bc8-4843-abf6-0d46a2aedc22 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:05.370081 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27240d-5bc8-4843-abf6-0d46a2aedc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:05.397320 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ae27240d-5bc8-4843-abf6-0d46a2aedc22 None None] MFA Rules not processed for user `07653944f5874e02b54c9bc0869c39d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:05.398662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ae27240d-5bc8-4843-abf6-0d46a2aedc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:05.485853 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1261/2532] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:05.490677 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151d525-e103-41ca-b60d-ea03d6de17b0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:05.557597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151d525-e103-41ca-b60d-ea03d6de17b0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:05.563232 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0715a9-f1a4-4644-86cf-d02c2c3e9e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:05.563232 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0715a9-f1a4-4644-86cf-d02c2c3e9e77 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:05.563232 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0715a9-f1a4-4644-86cf-d02c2c3e9e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:05.563521 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0715a9-f1a4-4644-86cf-d02c2c3e9e77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:05.564110 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0715a9-f1a4-4644-86cf-d02c2c3e9e77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:05.566017 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0715a9-f1a4-4644-86cf-d02c2c3e9e77 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:05.592244 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27240d-5bc8-4843-abf6-0d46a2aedc22 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:05.597686 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ae27240d-5bc8-4843-abf6-0d46a2aedc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:05.608653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27240d-5bc8-4843-abf6-0d46a2aedc22 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:05.613389 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39d680-d2e8-4159-80b7-8837dbc0fb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:05.614257 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39d680-d2e8-4159-80b7-8837dbc0fb7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:05.614579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39d680-d2e8-4159-80b7-8837dbc0fb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:05.614883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39d680-d2e8-4159-80b7-8837dbc0fb7b admin admin] PATH_INFO: `/v3/projects/106010db0a25438092fc802cffb34384` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:05.616668 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39d680-d2e8-4159-80b7-8837dbc0fb7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=106010db0a25438092fc802cffb34384)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:05.618851 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39d680-d2e8-4159-80b7-8837dbc0fb7b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:05.693840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1272/2533] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:05 2022] POST /identity/v3/projects => generated 393 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:05.698129 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0715a9-f1a4-4644-86cf-d02c2c3e9e77 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:05.762025 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0715a9-f1a4-4644-86cf-d02c2c3e9e77 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:05.762989 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:05.771170 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:05.776144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ce39d680-d2e8-4159-80b7-8837dbc0fb7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '106010db0a25438092fc802cffb34384', 'request_id': 'req-ce39d680-d2e8-4159-80b7-8837dbc0fb7b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:05.822540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16dba2-a8e4-4c20-b8f7-6e2255c75bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:05.823219 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16dba2-a8e4-4c20-b8f7-6e2255c75bbf admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:05.823507 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16dba2-a8e4-4c20-b8f7-6e2255c75bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:05.823785 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16dba2-a8e4-4c20-b8f7-6e2255c75bbf admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:05.824491 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16dba2-a8e4-4c20-b8f7-6e2255c75bbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:05.826469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16dba2-a8e4-4c20-b8f7-6e2255c75bbf admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:05.869375 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ce39d680-d2e8-4159-80b7-8837dbc0fb7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '106010db0a25438092fc802cffb34384', 'request_id': 'req-ce39d680-d2e8-4159-80b7-8837dbc0fb7b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:05.875327 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ce39d680-d2e8-4159-80b7-8837dbc0fb7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '106010db0a25438092fc802cffb34384', 'request_id': 'req-ce39d680-d2e8-4159-80b7-8837dbc0fb7b'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:05.886935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ce39d680-d2e8-4159-80b7-8837dbc0fb7b admin admin] The token cache is being invalidate because project 106010db0a25438092fc802cffb34384 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:05.887223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ce39d680-d2e8-4159-80b7-8837dbc0fb7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:05.887873 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ce39d680-d2e8-4159-80b7-8837dbc0fb7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:05.889411 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1262/2534] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:05 2022] DELETE /identity/v3/projects/106010db0a25438092fc802cffb34384 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:05.895003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc00d3-e10a-4396-ab33-ede717a3f7ed None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:05.895296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc00d3-e10a-4396-ab33-ede717a3f7ed None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:05.895579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc00d3-e10a-4396-ab33-ede717a3f7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:05.905288 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1273/2535] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:05 2022] POST /identity/v3/users => generated 303 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:05.910193 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16dba2-a8e4-4c20-b8f7-6e2255c75bbf admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:05.912091 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ab16dba2-a8e4-4c20-b8f7-6e2255c75bbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:05.914729 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-31dc00d3-e10a-4396-ab33-ede717a3f7ed None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:05.915913 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-31dc00d3-e10a-4396-ab33-ede717a3f7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:05.924191 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1263/2536] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:05.924412 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16dba2-a8e4-4c20-b8f7-6e2255c75bbf admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:05.930030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c5239-4bbc-4b15-beb0-caefecf58d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:05.930974 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c5239-4bbc-4b15-beb0-caefecf58d2c admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:05.931265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c5239-4bbc-4b15-beb0-caefecf58d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:05.931545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c5239-4bbc-4b15-beb0-caefecf58d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:05.932273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c5239-4bbc-4b15-beb0-caefecf58d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:05.933912 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c5239-4bbc-4b15-beb0-caefecf58d2c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:05.936993 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd880b-4969-4bfe-98b2-8dc28f8551f2 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:05.937290 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd880b-4969-4bfe-98b2-8dc28f8551f2 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:05.937570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd880b-4969-4bfe-98b2-8dc28f8551f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:05.947847 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1274/2537] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:25:05.953995 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c5239-4bbc-4b15-beb0-caefecf58d2c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:05.956951 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-27bd880b-4969-4bfe-98b2-8dc28f8551f2 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:05.958142 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-27bd880b-4969-4bfe-98b2-8dc28f8551f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:05.959612 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c5239-4bbc-4b15-beb0-caefecf58d2c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:05.964263 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98771a21-ae8f-4a0d-8272-a5d6124f0350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:05.965131 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98771a21-ae8f-4a0d-8272-a5d6124f0350 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:05.965471 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98771a21-ae8f-4a0d-8272-a5d6124f0350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:05.965755 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98771a21-ae8f-4a0d-8272-a5d6124f0350 admin admin] PATH_INFO: `/v3/projects/2c0342ea9a094498829ae9ebc5f8c8d3/users/58afdb2ed5d14b32b2a0e55208385ee3/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:05.966880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1264/2538] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:05.971049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd880b-4969-4bfe-98b2-8dc28f8551f2 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:05.980167 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd880b-4969-4bfe-98b2-8dc28f8551f2 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:05.981663 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98771a21-ae8f-4a0d-8272-a5d6124f0350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c0342ea9a094498829ae9ebc5f8c8d3, user_id=58afdb2ed5d14b32b2a0e55208385ee3, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:05.984389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98771a21-ae8f-4a0d-8272-a5d6124f0350 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:05.985018 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99653aa-6275-4d06-bec9-9a05731700b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:05.985671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99653aa-6275-4d06-bec9-9a05731700b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:05.985956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99653aa-6275-4d06-bec9-9a05731700b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:05.986237 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99653aa-6275-4d06-bec9-9a05731700b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:05.986992 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99653aa-6275-4d06-bec9-9a05731700b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:25:05.987360 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99653aa-6275-4d06-bec9-9a05731700b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:05.989125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99653aa-6275-4d06-bec9-9a05731700b9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:06.001136 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1265/2539] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:25:06.005377 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99653aa-6275-4d06-bec9-9a05731700b9 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:06.015697 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99653aa-6275-4d06-bec9-9a05731700b9 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:06.020418 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d95182-7613-4b07-ba8c-ee7158a324a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:06.021029 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d95182-7613-4b07-ba8c-ee7158a324a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:06.021892 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d95182-7613-4b07-ba8c-ee7158a324a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:06.022300 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d95182-7613-4b07-ba8c-ee7158a324a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:06.023012 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d95182-7613-4b07-ba8c-ee7158a324a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:06.025135 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d95182-7613-4b07-ba8c-ee7158a324a4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:06.092835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1275/2540] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:05 2022] PUT /identity/v3/projects/2c0342ea9a094498829ae9ebc5f8c8d3/users/58afdb2ed5d14b32b2a0e55208385ee3/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:06.096981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98771a21-ae8f-4a0d-8272-a5d6124f0350 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:06.143628 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1266/2541] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:06 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:06.144725 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d95182-7613-4b07-ba8c-ee7158a324a4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:06.162764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98771a21-ae8f-4a0d-8272-a5d6124f0350 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:06.212889 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d95182-7613-4b07-ba8c-ee7158a324a4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:06.217881 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044fbc7-a0ac-4437-96e9-a1b0ee3fd3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:06.218604 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044fbc7-a0ac-4437-96e9-a1b0ee3fd3fe admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:06.218892 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044fbc7-a0ac-4437-96e9-a1b0ee3fd3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:06.219167 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044fbc7-a0ac-4437-96e9-a1b0ee3fd3fe admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:06.219866 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044fbc7-a0ac-4437-96e9-a1b0ee3fd3fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:06.221797 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044fbc7-a0ac-4437-96e9-a1b0ee3fd3fe admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:06.226479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d687dc9-a841-437b-aa85-d35acb034460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:06.227151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d687dc9-a841-437b-aa85-d35acb034460 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:06.227435 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d687dc9-a841-437b-aa85-d35acb034460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:06.227716 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d687dc9-a841-437b-aa85-d35acb034460 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:06.228443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d687dc9-a841-437b-aa85-d35acb034460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:06.230029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d687dc9-a841-437b-aa85-d35acb034460 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:06.240478 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1276/2542] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:25:06.249044 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d687dc9-a841-437b-aa85-d35acb034460 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:06.259718 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d687dc9-a841-437b-aa85-d35acb034460 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:06.264477 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3311d13-d877-4b71-b07a-3d67ae21be74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:06.265340 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3311d13-d877-4b71-b07a-3d67ae21be74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:06.265631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3311d13-d877-4b71-b07a-3d67ae21be74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:06.265919 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3311d13-d877-4b71-b07a-3d67ae21be74 admin admin] PATH_INFO: `/v3/projects/2c0342ea9a094498829ae9ebc5f8c8d3/users/58afdb2ed5d14b32b2a0e55208385ee3/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:06.267907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3311d13-d877-4b71-b07a-3d67ae21be74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c0342ea9a094498829ae9ebc5f8c8d3, user_id=58afdb2ed5d14b32b2a0e55208385ee3, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:06.270595 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3311d13-d877-4b71-b07a-3d67ae21be74 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:06.280582 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:06.281533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1277/2543] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:06 2022] PUT /identity/v3/projects/2c0342ea9a094498829ae9ebc5f8c8d3/users/58afdb2ed5d14b32b2a0e55208385ee3/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:06.291332 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bd86f-f9d3-4846-b45e-929c7be9e4d0 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:06.291636 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bd86f-f9d3-4846-b45e-929c7be9e4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:06.291918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bd86f-f9d3-4846-b45e-929c7be9e4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:06.304397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1267/2544] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:06 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:06.308280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044fbc7-a0ac-4437-96e9-a1b0ee3fd3fe admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:06.317786 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044fbc7-a0ac-4437-96e9-a1b0ee3fd3fe admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:06.322739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-fe0bd86f-f9d3-4846-b45e-929c7be9e4d0 None None] MFA Rules not processed for user `58afdb2ed5d14b32b2a0e55208385ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:06.323962 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fe0bd86f-f9d3-4846-b45e-929c7be9e4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:06.375059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b926d5-0bd2-4e76-aad3-28d186d45319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:06.375732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b926d5-0bd2-4e76-aad3-28d186d45319 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:06.376014 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b926d5-0bd2-4e76-aad3-28d186d45319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:06.376323 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b926d5-0bd2-4e76-aad3-28d186d45319 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:06.377075 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b926d5-0bd2-4e76-aad3-28d186d45319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:06.378824 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b926d5-0bd2-4e76-aad3-28d186d45319 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:06.381806 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:06.390045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1268/2545] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:25:06.394027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b926d5-0bd2-4e76-aad3-28d186d45319 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:06.394691 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1278/2546] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:06.406195 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b926d5-0bd2-4e76-aad3-28d186d45319 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:06.406326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bd86f-f9d3-4846-b45e-929c7be9e4d0 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:06.406700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-fe0bd86f-f9d3-4846-b45e-929c7be9e4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:06.409692 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269c55e-99cb-4e3c-b760-3d162ee55757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:06.410570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269c55e-99cb-4e3c-b760-3d162ee55757 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:06.410965 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269c55e-99cb-4e3c-b760-3d162ee55757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:06.411325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269c55e-99cb-4e3c-b760-3d162ee55757 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:06.412159 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9269c55e-99cb-4e3c-b760-3d162ee55757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:06.414101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9269c55e-99cb-4e3c-b760-3d162ee55757 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:06.424272 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1269/2547] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:06.428947 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269c55e-99cb-4e3c-b760-3d162ee55757 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:06.438569 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269c55e-99cb-4e3c-b760-3d162ee55757 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:06.444109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70938a89-6a76-4b4d-8a66-24ceb8a7734c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:06.445071 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70938a89-6a76-4b4d-8a66-24ceb8a7734c admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:06.445357 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70938a89-6a76-4b4d-8a66-24ceb8a7734c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:06.445633 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70938a89-6a76-4b4d-8a66-24ceb8a7734c admin admin] PATH_INFO: `/v3/projects/c8495d512a9a47368dbb2b585b385b71/users/a6ce7b9ff5da4e9eae1b7615eafb82aa/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:06.462106 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70938a89-6a76-4b4d-8a66-24ceb8a7734c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8495d512a9a47368dbb2b585b385b71, user_id=a6ce7b9ff5da4e9eae1b7615eafb82aa, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:06.468694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70938a89-6a76-4b4d-8a66-24ceb8a7734c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:06.471395 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bd86f-f9d3-4846-b45e-929c7be9e4d0 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:06.476943 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeed01e-6954-4710-9cfb-df19913e34d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:06.477606 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeed01e-6954-4710-9cfb-df19913e34d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:06.477895 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeed01e-6954-4710-9cfb-df19913e34d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:06.478177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeed01e-6954-4710-9cfb-df19913e34d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:06.488044 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeed01e-6954-4710-9cfb-df19913e34d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:06.492200 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeed01e-6954-4710-9cfb-df19913e34d3 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:06.502106 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1279/2548] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:25:06.549780 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeed01e-6954-4710-9cfb-df19913e34d3 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:06.558216 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeed01e-6954-4710-9cfb-df19913e34d3 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:06.562188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75b882-986d-4994-b474-ca5c76de3b48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:06.563331 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75b882-986d-4994-b474-ca5c76de3b48 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:06.563331 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75b882-986d-4994-b474-ca5c76de3b48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:06.563533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75b882-986d-4994-b474-ca5c76de3b48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:06.568780 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-8f75b882-986d-4994-b474-ca5c76de3b48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:06.574090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1270/2549] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:06 2022] PUT /identity/v3/projects/c8495d512a9a47368dbb2b585b385b71/users/a6ce7b9ff5da4e9eae1b7615eafb82aa/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:06.598512 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8584615-dc38-41bc-82c8-dfb92a1767a5 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:06.599182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8584615-dc38-41bc-82c8-dfb92a1767a5 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:06.599924 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8584615-dc38-41bc-82c8-dfb92a1767a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:06.636049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e8584615-dc38-41bc-82c8-dfb92a1767a5 None None] MFA Rules not processed for user `a6ce7b9ff5da4e9eae1b7615eafb82aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:06.637293 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e8584615-dc38-41bc-82c8-dfb92a1767a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:06.647003 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75b882-986d-4994-b474-ca5c76de3b48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:06.648753 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75b882-986d-4994-b474-ca5c76de3b48 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:06.658530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1280/2550] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:06 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:25:06.705668 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1271/2551] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:06.711565 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75b882-986d-4994-b474-ca5c76de3b48 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:06.722393 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75b882-986d-4994-b474-ca5c76de3b48 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:06.745206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=100 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:06.786710 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2580c-c472-4bdf-be3c-51e59727858c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:06.787396 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2580c-c472-4bdf-be3c-51e59727858c admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:06.787681 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2580c-c472-4bdf-be3c-51e59727858c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:06.787959 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2580c-c472-4bdf-be3c-51e59727858c admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:06.788649 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2580c-c472-4bdf-be3c-51e59727858c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:06.790587 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b2580c-c472-4bdf-be3c-51e59727858c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:06.918558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1281/2552] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:06 2022] POST /identity/v3/projects => generated 385 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:06.920666 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8584615-dc38-41bc-82c8-dfb92a1767a5 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:06.932862 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8584615-dc38-41bc-82c8-dfb92a1767a5 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:06.989511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405baa51-ed20-42a9-b2ed-abfa89e0e716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:06.990750 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405baa51-ed20-42a9-b2ed-abfa89e0e716 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:06.991073 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405baa51-ed20-42a9-b2ed-abfa89e0e716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:06.991336 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405baa51-ed20-42a9-b2ed-abfa89e0e716 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:06.992028 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405baa51-ed20-42a9-b2ed-abfa89e0e716 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:06.993971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405baa51-ed20-42a9-b2ed-abfa89e0e716 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:07.095137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1272/2553] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:06 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:07.096412 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2580c-c472-4bdf-be3c-51e59727858c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:07.106763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2580c-c472-4bdf-be3c-51e59727858c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:07.111408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458d89d-e929-410e-acce-1343abe7245a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:07.112057 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458d89d-e929-410e-acce-1343abe7245a admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:07.112340 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458d89d-e929-410e-acce-1343abe7245a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:07.112617 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458d89d-e929-410e-acce-1343abe7245a admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:07.113348 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458d89d-e929-410e-acce-1343abe7245a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:07.115505 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458d89d-e929-410e-acce-1343abe7245a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:07.126040 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1282/2554] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:07.129966 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405baa51-ed20-42a9-b2ed-abfa89e0e716 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:07.146996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405baa51-ed20-42a9-b2ed-abfa89e0e716 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:07.146996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f77314-2b33-465c-929a-3c39964751e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:07.146996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f77314-2b33-465c-929a-3c39964751e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:07.147292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f77314-2b33-465c-929a-3c39964751e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:07.147403 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f77314-2b33-465c-929a-3c39964751e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:07.148127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f77314-2b33-465c-929a-3c39964751e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:07.149991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f77314-2b33-465c-929a-3c39964751e3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:07.160631 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1273/2555] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:07.164737 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458d89d-e929-410e-acce-1343abe7245a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:07.179834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458d89d-e929-410e-acce-1343abe7245a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:07.184341 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab940ca0-0f3a-44d0-ae31-d219a3bd10e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:07.185227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab940ca0-0f3a-44d0-ae31-d219a3bd10e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:07.185513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab940ca0-0f3a-44d0-ae31-d219a3bd10e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:07.185791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab940ca0-0f3a-44d0-ae31-d219a3bd10e6 admin admin] PATH_INFO: `/v3/projects/6005f6f09c8c4fc99e7d23f795373265/users/d9dd8b6cea1d46fea1543786ab978372/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:07.201242 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab940ca0-0f3a-44d0-ae31-d219a3bd10e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6005f6f09c8c4fc99e7d23f795373265, user_id=d9dd8b6cea1d46fea1543786ab978372, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:07.203936 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab940ca0-0f3a-44d0-ae31-d219a3bd10e6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:07.298443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1283/2556] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:07 2022] PUT /identity/v3/projects/6005f6f09c8c4fc99e7d23f795373265/users/d9dd8b6cea1d46fea1543786ab978372/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:07.309380 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3087a60-6d8a-499f-8b04-540451519e27 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:07.309677 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3087a60-6d8a-499f-8b04-540451519e27 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:07.309962 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3087a60-6d8a-499f-8b04-540451519e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:07.337721 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d3087a60-6d8a-499f-8b04-540451519e27 None None] MFA Rules not processed for user `d9dd8b6cea1d46fea1543786ab978372`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:07.339039 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d3087a60-6d8a-499f-8b04-540451519e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:07.360342 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:07.406867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1274/2557] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:07.411318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab940ca0-0f3a-44d0-ae31-d219a3bd10e6 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:07.477198 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab940ca0-0f3a-44d0-ae31-d219a3bd10e6 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:07.482313 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40489101-cae7-4686-840e-e9ac4f871666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:07.483009 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40489101-cae7-4686-840e-e9ac4f871666 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:07.483296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40489101-cae7-4686-840e-e9ac4f871666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:07.483574 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40489101-cae7-4686-840e-e9ac4f871666 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:07.484356 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40489101-cae7-4686-840e-e9ac4f871666 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:07.486336 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40489101-cae7-4686-840e-e9ac4f871666 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:07.595236 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1284/2558] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:07 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:07.599698 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3087a60-6d8a-499f-8b04-540451519e27 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:07.610984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3087a60-6d8a-499f-8b04-540451519e27 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:07.687210 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fa3e2-91ac-49d7-8490-1522efbe1819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:07.690405 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fa3e2-91ac-49d7-8490-1522efbe1819 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:07.690405 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fa3e2-91ac-49d7-8490-1522efbe1819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:07.690405 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fa3e2-91ac-49d7-8490-1522efbe1819 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:07.690405 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6fa3e2-91ac-49d7-8490-1522efbe1819 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:07.695389 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6fa3e2-91ac-49d7-8490-1522efbe1819 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:07.805762 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1275/2559] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:07 2022] POST /identity/v3/users => generated 299 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:07.810314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40489101-cae7-4686-840e-e9ac4f871666 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:07.824858 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40489101-cae7-4686-840e-e9ac4f871666 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:07.829683 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a437e40-6989-4e31-b88e-fed0b26eea9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:07.830349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a437e40-6989-4e31-b88e-fed0b26eea9d admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:07.830671 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a437e40-6989-4e31-b88e-fed0b26eea9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:07.830956 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a437e40-6989-4e31-b88e-fed0b26eea9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:07.831759 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a437e40-6989-4e31-b88e-fed0b26eea9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:07.833393 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a437e40-6989-4e31-b88e-fed0b26eea9d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:07.850236 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1285/2560] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:07.862105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fa3e2-91ac-49d7-8490-1522efbe1819 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:07.862105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fa3e2-91ac-49d7-8490-1522efbe1819 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:07.867726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b3d48-1a77-4509-af78-c5373f1ff757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:07.868621 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b3d48-1a77-4509-af78-c5373f1ff757 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:07.868917 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b3d48-1a77-4509-af78-c5373f1ff757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:07.869198 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b3d48-1a77-4509-af78-c5373f1ff757 admin admin] PATH_INFO: `/v3/projects/223b1389dfb44aa08a57e2a1959ac9dd/users/9a473c453c164bb0b0ab3da321510afa/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:07.886570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b3d48-1a77-4509-af78-c5373f1ff757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=223b1389dfb44aa08a57e2a1959ac9dd, user_id=9a473c453c164bb0b0ab3da321510afa, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:07.889332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b3d48-1a77-4509-af78-c5373f1ff757 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:07.983083 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1276/2561] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:07 2022] PUT /identity/v3/projects/223b1389dfb44aa08a57e2a1959ac9dd/users/9a473c453c164bb0b0ab3da321510afa/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:07.984649 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a437e40-6989-4e31-b88e-fed0b26eea9d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:08.007791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a437e40-6989-4e31-b88e-fed0b26eea9d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:08.070904 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:08.077513 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b3d48-1a77-4509-af78-c5373f1ff757 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:08.084971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-7a5b3d48-1a77-4509-af78-c5373f1ff757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:08.095996 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5429f11-8e66-444d-9715-986ee88373ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:08.096729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5429f11-8e66-444d-9715-986ee88373ea admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:08.097016 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5429f11-8e66-444d-9715-986ee88373ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:08.097295 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5429f11-8e66-444d-9715-986ee88373ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:08.098016 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5429f11-8e66-444d-9715-986ee88373ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:08.100757 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5429f11-8e66-444d-9715-986ee88373ea admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:08.112262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1286/2562] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:08.117167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5429f11-8e66-444d-9715-986ee88373ea admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:08.128134 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5429f11-8e66-444d-9715-986ee88373ea admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:08.133740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509736f-07fd-49c3-8c68-d2e74f53e7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:08.134769 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509736f-07fd-49c3-8c68-d2e74f53e7d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:08.135155 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509736f-07fd-49c3-8c68-d2e74f53e7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:08.135539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509736f-07fd-49c3-8c68-d2e74f53e7d3 admin admin] PATH_INFO: `/v3/projects/223b1389dfb44aa08a57e2a1959ac9dd/users/9a473c453c164bb0b0ab3da321510afa/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:08.137685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6509736f-07fd-49c3-8c68-d2e74f53e7d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=223b1389dfb44aa08a57e2a1959ac9dd, user_id=9a473c453c164bb0b0ab3da321510afa, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:08.140784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6509736f-07fd-49c3-8c68-d2e74f53e7d3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:08.153831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b3d48-1a77-4509-af78-c5373f1ff757 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:08.161206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1287/2563] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:08 2022] PUT /identity/v3/projects/223b1389dfb44aa08a57e2a1959ac9dd/users/9a473c453c164bb0b0ab3da321510afa/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:08.163547 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147379d-3774-4b1a-adb8-40b9190c4185 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:08.163862 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147379d-3774-4b1a-adb8-40b9190c4185 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:08.164115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147379d-3774-4b1a-adb8-40b9190c4185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:08.201249 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:08.208357 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3147379d-3774-4b1a-adb8-40b9190c4185 None None] MFA Rules not processed for user `9a473c453c164bb0b0ab3da321510afa`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:08.210127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3147379d-3774-4b1a-adb8-40b9190c4185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:08.211038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e56b4a-ac2f-4739-a241-e8cfbdd4b8f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:08.211691 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e56b4a-ac2f-4739-a241-e8cfbdd4b8f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:08.216246 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e56b4a-ac2f-4739-a241-e8cfbdd4b8f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:08.216246 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e56b4a-ac2f-4739-a241-e8cfbdd4b8f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:08.216246 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-34e56b4a-ac2f-4739-a241-e8cfbdd4b8f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:08.295016 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1288/2564] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:08 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:08.306009 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147379d-3774-4b1a-adb8-40b9190c4185 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:08.307447 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3147379d-3774-4b1a-adb8-40b9190c4185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:08.307780 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e56b4a-ac2f-4739-a241-e8cfbdd4b8f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:08.309290 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e56b4a-ac2f-4739-a241-e8cfbdd4b8f0 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:08.321220 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1277/2565] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:08.371462 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147379d-3774-4b1a-adb8-40b9190c4185 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:08.376603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b397c-d55b-4cee-a9f0-c95a3a3a5930 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:08.377360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b397c-d55b-4cee-a9f0-c95a3a3a5930 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:08.377739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b397c-d55b-4cee-a9f0-c95a3a3a5930 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:08.378015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b397c-d55b-4cee-a9f0-c95a3a3a5930 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:08.387441 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b397c-d55b-4cee-a9f0-c95a3a3a5930 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:08.388897 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b397c-d55b-4cee-a9f0-c95a3a3a5930 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:08.399550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1289/2566] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:08 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:25:09.433204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e56b4a-ac2f-4739-a241-e8cfbdd4b8f0 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:09.442290 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e56b4a-ac2f-4739-a241-e8cfbdd4b8f0 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:09.446934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0b963-4ea1-4151-97c7-236e38e05cf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:09.447583 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0b963-4ea1-4151-97c7-236e38e05cf4 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:09.447897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0b963-4ea1-4151-97c7-236e38e05cf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:09.448178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0b963-4ea1-4151-97c7-236e38e05cf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:09.508902 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0b963-4ea1-4151-97c7-236e38e05cf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:09.517002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0b963-4ea1-4151-97c7-236e38e05cf4 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:09.521653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1278/2567] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:14.227957 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b397c-d55b-4cee-a9f0-c95a3a3a5930 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:14.238669 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b397c-d55b-4cee-a9f0-c95a3a3a5930 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:14.244460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9f2ff-c6f5-4b1f-a435-059948b64d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:14.245321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9f2ff-c6f5-4b1f-a435-059948b64d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:14.245600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9f2ff-c6f5-4b1f-a435-059948b64d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:14.246400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9f2ff-c6f5-4b1f-a435-059948b64d18 admin admin] PATH_INFO: `/v3/users/07653944f5874e02b54c9bc0869c39d1` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:14.247086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a9f2ff-c6f5-4b1f-a435-059948b64d18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07653944f5874e02b54c9bc0869c39d1)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:14.249009 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a9f2ff-c6f5-4b1f-a435-059948b64d18 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:14.466211 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-98a9f2ff-c6f5-4b1f-a435-059948b64d18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07653944f5874e02b54c9bc0869c39d1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:14.473354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:14.480561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-98a9f2ff-c6f5-4b1f-a435-059948b64d18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07653944f5874e02b54c9bc0869c39d1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:14.481341 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-98a9f2ff-c6f5-4b1f-a435-059948b64d18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07653944f5874e02b54c9bc0869c39d1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:14.482053 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-98a9f2ff-c6f5-4b1f-a435-059948b64d18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07653944f5874e02b54c9bc0869c39d1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:14.496345 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-98a9f2ff-c6f5-4b1f-a435-059948b64d18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07653944f5874e02b54c9bc0869c39d1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:14.571518 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1290/2568] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:14 2022] DELETE /identity/v3/users/07653944f5874e02b54c9bc0869c39d1 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:14.576364 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdcb9c-6db4-487c-b493-95913bcb0ca1 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:14.576831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdcb9c-6db4-487c-b493-95913bcb0ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:14.576930 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdcb9c-6db4-487c-b493-95913bcb0ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:14.594736 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-1dbdcb9c-6db4-487c-b493-95913bcb0ca1 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:14.595874 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1dbdcb9c-6db4-487c-b493-95913bcb0ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:14.658985 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1279/2569] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:14.669035 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9f2ff-c6f5-4b1f-a435-059948b64d18 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:14.670343 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-98a9f2ff-c6f5-4b1f-a435-059948b64d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:14.736931 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9f2ff-c6f5-4b1f-a435-059948b64d18 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:14.741731 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d60343-7f04-420c-b801-62ab0c60dcfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:14.742590 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d60343-7f04-420c-b801-62ab0c60dcfa service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:14.742982 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d60343-7f04-420c-b801-62ab0c60dcfa service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:14.743411 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d60343-7f04-420c-b801-62ab0c60dcfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:14.754903 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d60343-7f04-420c-b801-62ab0c60dcfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:14.756804 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d60343-7f04-420c-b801-62ab0c60dcfa service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:14.767554 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1291/2570] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:15.378770 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbdcb9c-6db4-487c-b493-95913bcb0ca1 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:15.380224 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1dbdcb9c-6db4-487c-b493-95913bcb0ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:15.392426 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbdcb9c-6db4-487c-b493-95913bcb0ca1 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:15.418553 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b37ec-5caf-42ff-ab61-dbc70aa11d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:15.419495 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b37ec-5caf-42ff-ab61-dbc70aa11d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:15.419851 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b37ec-5caf-42ff-ab61-dbc70aa11d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:15.419976 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b37ec-5caf-42ff-ab61-dbc70aa11d0e admin admin] PATH_INFO: `/v3/projects/7b23bd7324194a7ab0438e13dcf70c4b` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:15.421235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b37ec-5caf-42ff-ab61-dbc70aa11d0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b23bd7324194a7ab0438e13dcf70c4b)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:15.423275 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235b37ec-5caf-42ff-ab61-dbc70aa11d0e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:15.539877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:15.573025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-235b37ec-5caf-42ff-ab61-dbc70aa11d0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b23bd7324194a7ab0438e13dcf70c4b', 'request_id': 'req-235b37ec-5caf-42ff-ab61-dbc70aa11d0e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:15.640958 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-235b37ec-5caf-42ff-ab61-dbc70aa11d0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b23bd7324194a7ab0438e13dcf70c4b', 'request_id': 'req-235b37ec-5caf-42ff-ab61-dbc70aa11d0e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:15.649070 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-235b37ec-5caf-42ff-ab61-dbc70aa11d0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b23bd7324194a7ab0438e13dcf70c4b', 'request_id': 'req-235b37ec-5caf-42ff-ab61-dbc70aa11d0e'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:15.664254 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-235b37ec-5caf-42ff-ab61-dbc70aa11d0e admin admin] The token cache is being invalidate because project 7b23bd7324194a7ab0438e13dcf70c4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:15.665850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-235b37ec-5caf-42ff-ab61-dbc70aa11d0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:15.666677 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-235b37ec-5caf-42ff-ab61-dbc70aa11d0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:15.669016 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1280/2571] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:15 2022] DELETE /identity/v3/projects/7b23bd7324194a7ab0438e13dcf70c4b => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:15.673889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d60343-7f04-420c-b801-62ab0c60dcfa service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:15.684330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d60343-7f04-420c-b801-62ab0c60dcfa service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:15.750564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabfe5e-95da-4c7c-96c1-2ef70dd08a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:15.751542 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabfe5e-95da-4c7c-96c1-2ef70dd08a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:15.751898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabfe5e-95da-4c7c-96c1-2ef70dd08a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:15.752395 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabfe5e-95da-4c7c-96c1-2ef70dd08a3c admin admin] PATH_INFO: `/v3/users/58afdb2ed5d14b32b2a0e55208385ee3` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:15.753766 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabfe5e-95da-4c7c-96c1-2ef70dd08a3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=58afdb2ed5d14b32b2a0e55208385ee3)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:15.756672 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabfe5e-95da-4c7c-96c1-2ef70dd08a3c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:16.034193 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3fabfe5e-95da-4c7c-96c1-2ef70dd08a3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58afdb2ed5d14b32b2a0e55208385ee3'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:16.058640 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3fabfe5e-95da-4c7c-96c1-2ef70dd08a3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58afdb2ed5d14b32b2a0e55208385ee3'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:16.059393 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3fabfe5e-95da-4c7c-96c1-2ef70dd08a3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58afdb2ed5d14b32b2a0e55208385ee3'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:16.060027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3fabfe5e-95da-4c7c-96c1-2ef70dd08a3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58afdb2ed5d14b32b2a0e55208385ee3'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:16.068660 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-3fabfe5e-95da-4c7c-96c1-2ef70dd08a3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58afdb2ed5d14b32b2a0e55208385ee3'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:16.071637 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:16.184773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1292/2572] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:15 2022] DELETE /identity/v3/users/58afdb2ed5d14b32b2a0e55208385ee3 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:16.194454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b37ec-5caf-42ff-ab61-dbc70aa11d0e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:16.196109 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-235b37ec-5caf-42ff-ab61-dbc70aa11d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:16.210480 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235b37ec-5caf-42ff-ab61-dbc70aa11d0e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:16.273884 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0630eeb-4207-4916-b374-47351346697f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:16.275094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0630eeb-4207-4916-b374-47351346697f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:16.275380 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0630eeb-4207-4916-b374-47351346697f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:16.275658 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0630eeb-4207-4916-b374-47351346697f admin admin] PATH_INFO: `/v3/users/f3821e46c09b449d85c46c58d99bf607` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:16.277054 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0630eeb-4207-4916-b374-47351346697f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3821e46c09b449d85c46c58d99bf607)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:16.279217 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0630eeb-4207-4916-b374-47351346697f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:16.477873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabfe5e-95da-4c7c-96c1-2ef70dd08a3c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:16.479738 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3fabfe5e-95da-4c7c-96c1-2ef70dd08a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:16.504433 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabfe5e-95da-4c7c-96c1-2ef70dd08a3c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:16.507196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a0630eeb-4207-4916-b374-47351346697f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3821e46c09b449d85c46c58d99bf607'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:16.510395 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21b190-ced5-42b8-8c5a-d22ade702d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:16.511438 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21b190-ced5-42b8-8c5a-d22ade702d27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:16.511823 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21b190-ced5-42b8-8c5a-d22ade702d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:16.512210 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21b190-ced5-42b8-8c5a-d22ade702d27 admin admin] PATH_INFO: `/v3/users/c51a2c1727f745e2a6068597015a7a06` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:16.513601 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21b190-ced5-42b8-8c5a-d22ade702d27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c51a2c1727f745e2a6068597015a7a06)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:16.515866 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e21b190-ced5-42b8-8c5a-d22ade702d27 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:16.524178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a0630eeb-4207-4916-b374-47351346697f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3821e46c09b449d85c46c58d99bf607'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:16.596283 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a0630eeb-4207-4916-b374-47351346697f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3821e46c09b449d85c46c58d99bf607'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:16.597324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a0630eeb-4207-4916-b374-47351346697f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3821e46c09b449d85c46c58d99bf607'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:16.597949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a0630eeb-4207-4916-b374-47351346697f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3821e46c09b449d85c46c58d99bf607'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:16.613349 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1281/2573] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:16 2022] DELETE /identity/v3/users/f3821e46c09b449d85c46c58d99bf607 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:16.716043 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0630eeb-4207-4916-b374-47351346697f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:16.717641 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a0630eeb-4207-4916-b374-47351346697f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:16.730167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0e21b190-ced5-42b8-8c5a-d22ade702d27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c51a2c1727f745e2a6068597015a7a06'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:16.788631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0e21b190-ced5-42b8-8c5a-d22ade702d27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c51a2c1727f745e2a6068597015a7a06'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:16.790742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0e21b190-ced5-42b8-8c5a-d22ade702d27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c51a2c1727f745e2a6068597015a7a06'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:16.794298 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0e21b190-ced5-42b8-8c5a-d22ade702d27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c51a2c1727f745e2a6068597015a7a06'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:16.801834 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-0e21b190-ced5-42b8-8c5a-d22ade702d27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c51a2c1727f745e2a6068597015a7a06'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:16.814117 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:16.854082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0630eeb-4207-4916-b374-47351346697f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:16.861173 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a487fc9-3cbb-4d45-a536-56336e585cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:16.862486 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a487fc9-3cbb-4d45-a536-56336e585cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:16.862791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a487fc9-3cbb-4d45-a536-56336e585cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:16.863067 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a487fc9-3cbb-4d45-a536-56336e585cd4 admin admin] PATH_INFO: `/v3/projects/2c0342ea9a094498829ae9ebc5f8c8d3` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:16.864301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a487fc9-3cbb-4d45-a536-56336e585cd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c0342ea9a094498829ae9ebc5f8c8d3)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:16.867381 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a487fc9-3cbb-4d45-a536-56336e585cd4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:16.942002 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1293/2574] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:16 2022] DELETE /identity/v3/users/c51a2c1727f745e2a6068597015a7a06 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:16.942002 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21b190-ced5-42b8-8c5a-d22ade702d27 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:16.943737 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0e21b190-ced5-42b8-8c5a-d22ade702d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:17.014895 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e21b190-ced5-42b8-8c5a-d22ade702d27 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:17.019591 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1a487fc9-3cbb-4d45-a536-56336e585cd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c0342ea9a094498829ae9ebc5f8c8d3', 'request_id': 'req-1a487fc9-3cbb-4d45-a536-56336e585cd4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:17.081791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a22184-c3c7-44e0-af4a-6bbd9f18b554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:17.082724 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a22184-c3c7-44e0-af4a-6bbd9f18b554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:17.083032 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a22184-c3c7-44e0-af4a-6bbd9f18b554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:17.084216 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a22184-c3c7-44e0-af4a-6bbd9f18b554 admin admin] PATH_INFO: `/v3/projects/7e75e404ef4f4118a95ca8fb8d98becd` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:17.085203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a22184-c3c7-44e0-af4a-6bbd9f18b554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e75e404ef4f4118a95ca8fb8d98becd)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:17.087202 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a22184-c3c7-44e0-af4a-6bbd9f18b554 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:17.102904 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:17.108887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1a487fc9-3cbb-4d45-a536-56336e585cd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c0342ea9a094498829ae9ebc5f8c8d3', 'request_id': 'req-1a487fc9-3cbb-4d45-a536-56336e585cd4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:17.115270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1a487fc9-3cbb-4d45-a536-56336e585cd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c0342ea9a094498829ae9ebc5f8c8d3', 'request_id': 'req-1a487fc9-3cbb-4d45-a536-56336e585cd4'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:17.126231 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1a487fc9-3cbb-4d45-a536-56336e585cd4 admin admin] The token cache is being invalidate because project 2c0342ea9a094498829ae9ebc5f8c8d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:17.126555 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1a487fc9-3cbb-4d45-a536-56336e585cd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:17.127204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1a487fc9-3cbb-4d45-a536-56336e585cd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:17.128741 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1282/2575] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:16 2022] DELETE /identity/v3/projects/2c0342ea9a094498829ae9ebc5f8c8d3 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:17.129961 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6af44-e45c-4d40-a170-64039964217e None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:17.130873 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6af44-e45c-4d40-a170-64039964217e None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:17.131131 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6af44-e45c-4d40-a170-64039964217e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:17.148653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-bcb6af44-e45c-4d40-a170-64039964217e None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:17.150100 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bcb6af44-e45c-4d40-a170-64039964217e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:17.157737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1283/2576] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:17.158548 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6af44-e45c-4d40-a170-64039964217e None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:17.160008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bcb6af44-e45c-4d40-a170-64039964217e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:17.221144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6af44-e45c-4d40-a170-64039964217e None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:17.225159 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6e246-7040-4bfa-9f35-e741ac438ef9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:17.225827 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6e246-7040-4bfa-9f35-e741ac438ef9 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:17.226101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6e246-7040-4bfa-9f35-e741ac438ef9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:17.226406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6e246-7040-4bfa-9f35-e741ac438ef9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:17.229653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e8b6e246-7040-4bfa-9f35-e741ac438ef9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:17.248330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-86a22184-c3c7-44e0-af4a-6bbd9f18b554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e75e404ef4f4118a95ca8fb8d98becd', 'request_id': 'req-86a22184-c3c7-44e0-af4a-6bbd9f18b554'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:17.254519 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-86a22184-c3c7-44e0-af4a-6bbd9f18b554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e75e404ef4f4118a95ca8fb8d98becd', 'request_id': 'req-86a22184-c3c7-44e0-af4a-6bbd9f18b554'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:17.260779 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-86a22184-c3c7-44e0-af4a-6bbd9f18b554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e75e404ef4f4118a95ca8fb8d98becd', 'request_id': 'req-86a22184-c3c7-44e0-af4a-6bbd9f18b554'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:17.288867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b6e246-7040-4bfa-9f35-e741ac438ef9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:17.290303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b6e246-7040-4bfa-9f35-e741ac438ef9 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:17.299225 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1284/2577] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:17.300485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cf2d6-e5df-4aac-b364-b4d7520886a0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:17.300771 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cf2d6-e5df-4aac-b364-b4d7520886a0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:17.301056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cf2d6-e5df-4aac-b364-b4d7520886a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:17.319812 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-694cf2d6-e5df-4aac-b364-b4d7520886a0 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:17.321100 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-694cf2d6-e5df-4aac-b364-b4d7520886a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:17.333028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-86a22184-c3c7-44e0-af4a-6bbd9f18b554 admin admin] The token cache is being invalidate because project 7e75e404ef4f4118a95ca8fb8d98becd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:17.333400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-86a22184-c3c7-44e0-af4a-6bbd9f18b554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:17.334773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-86a22184-c3c7-44e0-af4a-6bbd9f18b554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:17.336626 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1294/2578] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:16 2022] DELETE /identity/v3/projects/7e75e404ef4f4118a95ca8fb8d98becd => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:17.337458 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a22184-c3c7-44e0-af4a-6bbd9f18b554 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:17.338891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-86a22184-c3c7-44e0-af4a-6bbd9f18b554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:17.390898 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1285/2579] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:17.391753 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cf2d6-e5df-4aac-b364-b4d7520886a0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:17.401332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-694cf2d6-e5df-4aac-b364-b4d7520886a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:17.413700 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cf2d6-e5df-4aac-b364-b4d7520886a0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:17.414359 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a22184-c3c7-44e0-af4a-6bbd9f18b554 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:17.418218 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6e2b5-5880-4192-8d55-31f1a369976e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:17.418905 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6e2b5-5880-4192-8d55-31f1a369976e service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:17.419292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6e2b5-5880-4192-8d55-31f1a369976e service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:17.419573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6e2b5-5880-4192-8d55-31f1a369976e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:17.419818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef05ef-7315-4225-9f7e-76c20a62a4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:17.420705 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef05ef-7315-4225-9f7e-76c20a62a4c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:17.421049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef05ef-7315-4225-9f7e-76c20a62a4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:17.421326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef05ef-7315-4225-9f7e-76c20a62a4c6 admin admin] PATH_INFO: `/v3/users/ba662104accd44a1aa5e7a473bb51557` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:17.421503 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a3d6e2b5-5880-4192-8d55-31f1a369976e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:17.422579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef05ef-7315-4225-9f7e-76c20a62a4c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba662104accd44a1aa5e7a473bb51557)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:17.424507 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef05ef-7315-4225-9f7e-76c20a62a4c6 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:17.435237 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6e2b5-5880-4192-8d55-31f1a369976e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:17.436753 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6e2b5-5880-4192-8d55-31f1a369976e service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:17.447381 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1295/2580] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:17.451824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646bf097-c2fa-40c7-815f-1a9f66b88373 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:17.452142 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646bf097-c2fa-40c7-815f-1a9f66b88373 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:17.452423 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646bf097-c2fa-40c7-815f-1a9f66b88373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:17.475764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-646bf097-c2fa-40c7-815f-1a9f66b88373 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:17.475764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-646bf097-c2fa-40c7-815f-1a9f66b88373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:17.485455 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1296/2581] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:17.490274 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646bf097-c2fa-40c7-815f-1a9f66b88373 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:17.500203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646bf097-c2fa-40c7-815f-1a9f66b88373 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:17.505082 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147ff39-f8d8-432a-82b8-32b543e04218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:17.506544 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147ff39-f8d8-432a-82b8-32b543e04218 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:17.506836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147ff39-f8d8-432a-82b8-32b543e04218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:17.507117 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147ff39-f8d8-432a-82b8-32b543e04218 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:17.507853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2147ff39-f8d8-432a-82b8-32b543e04218 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:25:17.508199 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2147ff39-f8d8-432a-82b8-32b543e04218 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:17.509908 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2147ff39-f8d8-432a-82b8-32b543e04218 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:17.527355 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1297/2582] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:25:17.528437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147ff39-f8d8-432a-82b8-32b543e04218 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:17.538361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147ff39-f8d8-432a-82b8-32b543e04218 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:17.543033 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df1cdd-0097-4d64-a0ad-98a2d4e74a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:17.543684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df1cdd-0097-4d64-a0ad-98a2d4e74a94 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:17.543965 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df1cdd-0097-4d64-a0ad-98a2d4e74a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:17.544242 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df1cdd-0097-4d64-a0ad-98a2d4e74a94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:17.544935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df1cdd-0097-4d64-a0ad-98a2d4e74a94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:17.546876 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df1cdd-0097-4d64-a0ad-98a2d4e74a94 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:17.619210 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-baef05ef-7315-4225-9f7e-76c20a62a4c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba662104accd44a1aa5e7a473bb51557'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:17.631650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-baef05ef-7315-4225-9f7e-76c20a62a4c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba662104accd44a1aa5e7a473bb51557'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:17.676790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1298/2583] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:17 2022] POST /identity/v3/projects => generated 389 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:17.681660 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df1cdd-0097-4d64-a0ad-98a2d4e74a94 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:17.762020 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df1cdd-0097-4d64-a0ad-98a2d4e74a94 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:17.770479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b941958-cdef-4a6c-bf84-f3bb0402ef4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:17.771904 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b941958-cdef-4a6c-bf84-f3bb0402ef4f admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:17.772323 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b941958-cdef-4a6c-bf84-f3bb0402ef4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:17.772573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b941958-cdef-4a6c-bf84-f3bb0402ef4f admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:17.773342 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b941958-cdef-4a6c-bf84-f3bb0402ef4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:17.776152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b941958-cdef-4a6c-bf84-f3bb0402ef4f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:17.860064 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-baef05ef-7315-4225-9f7e-76c20a62a4c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba662104accd44a1aa5e7a473bb51557'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:17.860734 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-baef05ef-7315-4225-9f7e-76c20a62a4c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba662104accd44a1aa5e7a473bb51557'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:17.861325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-baef05ef-7315-4225-9f7e-76c20a62a4c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba662104accd44a1aa5e7a473bb51557'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:17.877075 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1286/2584] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:17 2022] DELETE /identity/v3/users/ba662104accd44a1aa5e7a473bb51557 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:17.966772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:18.046004 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1299/2585] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:17 2022] POST /identity/v3/users => generated 295 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:18.047249 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef05ef-7315-4225-9f7e-76c20a62a4c6 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:18.048843 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-baef05ef-7315-4225-9f7e-76c20a62a4c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:18.067389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b941958-cdef-4a6c-bf84-f3bb0402ef4f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:18.069099 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4b941958-cdef-4a6c-bf84-f3bb0402ef4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:18.081868 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b941958-cdef-4a6c-bf84-f3bb0402ef4f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:18.120012 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef05ef-7315-4225-9f7e-76c20a62a4c6 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:18.124716 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc64e9-7ef8-4345-846b-788ac0ce1c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:18.125438 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc64e9-7ef8-4345-846b-788ac0ce1c02 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:18.125755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc64e9-7ef8-4345-846b-788ac0ce1c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:18.126035 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc64e9-7ef8-4345-846b-788ac0ce1c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:18.126817 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc64e9-7ef8-4345-846b-788ac0ce1c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:18.128462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc64e9-7ef8-4345-846b-788ac0ce1c02 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:18.142552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1287/2586] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:18.145289 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc64e9-7ef8-4345-846b-788ac0ce1c02 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:18.149737 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f69a8-e1d9-4025-bc70-db341d82d47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:18.150947 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f69a8-e1d9-4025-bc70-db341d82d47b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:18.151229 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f69a8-e1d9-4025-bc70-db341d82d47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:18.151505 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f69a8-e1d9-4025-bc70-db341d82d47b admin admin] PATH_INFO: `/v3/projects/eebba33fe87a4f32ba4f697eadf56bb8` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:18.152763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f69a8-e1d9-4025-bc70-db341d82d47b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eebba33fe87a4f32ba4f697eadf56bb8)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:18.154958 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f69a8-e1d9-4025-bc70-db341d82d47b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:18.157690 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc64e9-7ef8-4345-846b-788ac0ce1c02 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:18.164559 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99a982-2b6b-4a2b-b606-a0db67823d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:18.165477 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99a982-2b6b-4a2b-b606-a0db67823d45 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:18.165915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99a982-2b6b-4a2b-b606-a0db67823d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:18.166214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99a982-2b6b-4a2b-b606-a0db67823d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:18.167137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99a982-2b6b-4a2b-b606-a0db67823d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:18.168761 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99a982-2b6b-4a2b-b606-a0db67823d45 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:18.186755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1288/2587] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:18.205709 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99a982-2b6b-4a2b-b606-a0db67823d45 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:18.217082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99a982-2b6b-4a2b-b606-a0db67823d45 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:18.222069 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467dbc2b-917e-4a79-9c68-c0054c4fc666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:18.227569 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467dbc2b-917e-4a79-9c68-c0054c4fc666 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:18.227569 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467dbc2b-917e-4a79-9c68-c0054c4fc666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:18.227569 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467dbc2b-917e-4a79-9c68-c0054c4fc666 admin admin] PATH_INFO: `/v3/projects/c1a26c3444f240798c013d1b72d58794/users/300105986ab64ca69583b2a4305ad6c8/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:18.228267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:18.241419 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467dbc2b-917e-4a79-9c68-c0054c4fc666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1a26c3444f240798c013d1b72d58794, user_id=300105986ab64ca69583b2a4305ad6c8, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:18.244681 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467dbc2b-917e-4a79-9c68-c0054c4fc666 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:18.349612 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1289/2588] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:18 2022] PUT /identity/v3/projects/c1a26c3444f240798c013d1b72d58794/users/300105986ab64ca69583b2a4305ad6c8/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:18.350497 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467dbc2b-917e-4a79-9c68-c0054c4fc666 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:18.352424 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-467dbc2b-917e-4a79-9c68-c0054c4fc666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:18.358048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ca2f69a8-e1d9-4025-bc70-db341d82d47b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eebba33fe87a4f32ba4f697eadf56bb8', 'request_id': 'req-ca2f69a8-e1d9-4025-bc70-db341d82d47b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:18.365415 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ca2f69a8-e1d9-4025-bc70-db341d82d47b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eebba33fe87a4f32ba4f697eadf56bb8', 'request_id': 'req-ca2f69a8-e1d9-4025-bc70-db341d82d47b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:18.372796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ca2f69a8-e1d9-4025-bc70-db341d82d47b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eebba33fe87a4f32ba4f697eadf56bb8', 'request_id': 'req-ca2f69a8-e1d9-4025-bc70-db341d82d47b'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:18.421089 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467dbc2b-917e-4a79-9c68-c0054c4fc666 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:18.440449 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ca2f69a8-e1d9-4025-bc70-db341d82d47b admin admin] The token cache is being invalidate because project eebba33fe87a4f32ba4f697eadf56bb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:18.442485 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ca2f69a8-e1d9-4025-bc70-db341d82d47b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:18.442485 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-ca2f69a8-e1d9-4025-bc70-db341d82d47b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:18.443779 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1300/2589] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:18 2022] DELETE /identity/v3/projects/eebba33fe87a4f32ba4f697eadf56bb8 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:18.454394 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b77c30-fa00-4d3d-b285-415f5331f737 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:18.454394 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b77c30-fa00-4d3d-b285-415f5331f737 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:18.454394 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b77c30-fa00-4d3d-b285-415f5331f737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:18.479249 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-19b77c30-fa00-4d3d-b285-415f5331f737 None None] MFA Rules not processed for user `300105986ab64ca69583b2a4305ad6c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:18.480518 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-19b77c30-fa00-4d3d-b285-415f5331f737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:18.485843 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab4f77-6186-4551-b180-8f87063dd6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:18.487112 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab4f77-6186-4551-b180-8f87063dd6d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:18.487571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab4f77-6186-4551-b180-8f87063dd6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:18.488069 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab4f77-6186-4551-b180-8f87063dd6d9 admin admin] PATH_INFO: `/v3/projects/34ebafeecaa04e3196f815aff6a6f00d` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:18.490281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab4f77-6186-4551-b180-8f87063dd6d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34ebafeecaa04e3196f815aff6a6f00d)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:18.492727 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab4f77-6186-4551-b180-8f87063dd6d9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:18.559823 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1301/2590] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:18.564467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b77c30-fa00-4d3d-b285-415f5331f737 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:18.566282 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-19b77c30-fa00-4d3d-b285-415f5331f737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:18.581328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b77c30-fa00-4d3d-b285-415f5331f737 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:18.587408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508dde11-2656-4a55-9b58-b74cf7d49639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:18.588261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dde11-2656-4a55-9b58-b74cf7d49639 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:18.588546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dde11-2656-4a55-9b58-b74cf7d49639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:18.588825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dde11-2656-4a55-9b58-b74cf7d49639 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:18.589538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508dde11-2656-4a55-9b58-b74cf7d49639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:18.591616 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508dde11-2656-4a55-9b58-b74cf7d49639 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:18.636856 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c0ab4f77-6186-4551-b180-8f87063dd6d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34ebafeecaa04e3196f815aff6a6f00d', 'request_id': 'req-c0ab4f77-6186-4551-b180-8f87063dd6d9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:18.699846 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1302/2591] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:18 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:18.703699 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508dde11-2656-4a55-9b58-b74cf7d49639 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:18.743760 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c0ab4f77-6186-4551-b180-8f87063dd6d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34ebafeecaa04e3196f815aff6a6f00d', 'request_id': 'req-c0ab4f77-6186-4551-b180-8f87063dd6d9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:18.750078 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c0ab4f77-6186-4551-b180-8f87063dd6d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34ebafeecaa04e3196f815aff6a6f00d', 'request_id': 'req-c0ab4f77-6186-4551-b180-8f87063dd6d9'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:18.762047 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c0ab4f77-6186-4551-b180-8f87063dd6d9 admin admin] The token cache is being invalidate because project 34ebafeecaa04e3196f815aff6a6f00d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:18.762339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c0ab4f77-6186-4551-b180-8f87063dd6d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:18.763026 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c0ab4f77-6186-4551-b180-8f87063dd6d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:18.764636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1290/2592] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:18 2022] DELETE /identity/v3/projects/34ebafeecaa04e3196f815aff6a6f00d => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:18.764996 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508dde11-2656-4a55-9b58-b74cf7d49639 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:18.778133 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8ad4f-bd15-4cff-83c9-10d87d946b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:18.778874 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8ad4f-bd15-4cff-83c9-10d87d946b8b admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:18.779161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8ad4f-bd15-4cff-83c9-10d87d946b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:18.779439 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8ad4f-bd15-4cff-83c9-10d87d946b8b admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:18.780150 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8ad4f-bd15-4cff-83c9-10d87d946b8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:18.782282 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb8ad4f-bd15-4cff-83c9-10d87d946b8b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:18.908565 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1303/2593] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:18 2022] POST /identity/v3/users => generated 299 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:18.913205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab4f77-6186-4551-b180-8f87063dd6d9 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:18.914841 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c0ab4f77-6186-4551-b180-8f87063dd6d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:18.988613 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab4f77-6186-4551-b180-8f87063dd6d9 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:18.996362 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8ba4b-c3e8-46b1-9e16-8ef5871f6bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:18.997060 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8ba4b-c3e8-46b1-9e16-8ef5871f6bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:18.997551 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8ba4b-c3e8-46b1-9e16-8ef5871f6bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:18.997745 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d8ba4b-c3e8-46b1-9e16-8ef5871f6bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:18.998556 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d8ba4b-c3e8-46b1-9e16-8ef5871f6bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:19.003095 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d8ba4b-c3e8-46b1-9e16-8ef5871f6bb4 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:19.014508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1291/2594] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:19.016037 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8ad4f-bd15-4cff-83c9-10d87d946b8b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:19.030606 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb8ad4f-bd15-4cff-83c9-10d87d946b8b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:19.038637 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf3eac-e25f-4ad4-9b4d-52472e0c26b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:19.038637 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf3eac-e25f-4ad4-9b4d-52472e0c26b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:19.038637 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf3eac-e25f-4ad4-9b4d-52472e0c26b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:19.038637 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf3eac-e25f-4ad4-9b4d-52472e0c26b4 admin admin] PATH_INFO: `/v3/projects/0513b9e88c5e4d69bceb5e9d1d5b8a37/users/fb653efd630245c39cba81bdda223b18/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:19.054339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf3eac-e25f-4ad4-9b4d-52472e0c26b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0513b9e88c5e4d69bceb5e9d1d5b8a37, user_id=fb653efd630245c39cba81bdda223b18, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:19.057899 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf3eac-e25f-4ad4-9b4d-52472e0c26b4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:19.132769 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:19.138654 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1304/2595] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:19 2022] PUT /identity/v3/projects/0513b9e88c5e4d69bceb5e9d1d5b8a37/users/fb653efd630245c39cba81bdda223b18/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:19.142615 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8ba4b-c3e8-46b1-9e16-8ef5871f6bb4 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:19.153309 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d8ba4b-c3e8-46b1-9e16-8ef5871f6bb4 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:19.210847 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d71b74-5cb6-4b16-b663-4be5af15a221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:19.211559 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d71b74-5cb6-4b16-b663-4be5af15a221 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:19.211858 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d71b74-5cb6-4b16-b663-4be5af15a221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:19.212133 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d71b74-5cb6-4b16-b663-4be5af15a221 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:19.212859 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d71b74-5cb6-4b16-b663-4be5af15a221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:19.214531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d71b74-5cb6-4b16-b663-4be5af15a221 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:19.228571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1292/2596] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:19.229483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:19.230739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf3eac-e25f-4ad4-9b4d-52472e0c26b4 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:19.241649 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf3eac-e25f-4ad4-9b4d-52472e0c26b4 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:19.246271 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2209f-1f84-4ef9-be9f-6c144bb37842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:19.247129 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2209f-1f84-4ef9-be9f-6c144bb37842 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:19.247477 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2209f-1f84-4ef9-be9f-6c144bb37842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:19.247763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2209f-1f84-4ef9-be9f-6c144bb37842 admin admin] PATH_INFO: `/v3/projects/0513b9e88c5e4d69bceb5e9d1d5b8a37/users/fb653efd630245c39cba81bdda223b18/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:19.249767 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d2209f-1f84-4ef9-be9f-6c144bb37842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0513b9e88c5e4d69bceb5e9d1d5b8a37, user_id=fb653efd630245c39cba81bdda223b18, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:19.252561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d2209f-1f84-4ef9-be9f-6c144bb37842 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:19.262983 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1305/2597] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:19 2022] PUT /identity/v3/projects/0513b9e88c5e4d69bceb5e9d1d5b8a37/users/fb653efd630245c39cba81bdda223b18/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:19.274520 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd1ce2-c0db-4bd4-839f-3d237193c342 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:19.274791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd1ce2-c0db-4bd4-839f-3d237193c342 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:19.275335 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd1ce2-c0db-4bd4-839f-3d237193c342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:19.304448 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a8dd1ce2-c0db-4bd4-839f-3d237193c342 None None] MFA Rules not processed for user `fb653efd630245c39cba81bdda223b18`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:19.305707 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a8dd1ce2-c0db-4bd4-839f-3d237193c342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:19.373641 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1293/2598] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:19.385512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2209f-1f84-4ef9-be9f-6c144bb37842 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:19.387248 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-60d2209f-1f84-4ef9-be9f-6c144bb37842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:19.444879 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d2209f-1f84-4ef9-be9f-6c144bb37842 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:19.448536 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018e701-a19e-49db-9a6a-d3e3333beedd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:19.449231 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018e701-a19e-49db-9a6a-d3e3333beedd service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:19.449450 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018e701-a19e-49db-9a6a-d3e3333beedd service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:19.449792 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018e701-a19e-49db-9a6a-d3e3333beedd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:19.460908 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2018e701-a19e-49db-9a6a-d3e3333beedd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:19.462355 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2018e701-a19e-49db-9a6a-d3e3333beedd service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:19.472138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1306/2599] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:25:19.603228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd1ce2-c0db-4bd4-839f-3d237193c342 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:19.665476 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd1ce2-c0db-4bd4-839f-3d237193c342 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:19.670050 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a689e-2271-4f3e-9e8d-bde405643edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:19.670954 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a689e-2271-4f3e-9e8d-bde405643edb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:19.671238 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a689e-2271-4f3e-9e8d-bde405643edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:19.671516 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a689e-2271-4f3e-9e8d-bde405643edb admin admin] PATH_INFO: `/v3/users/300105986ab64ca69583b2a4305ad6c8` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:19.672696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a689e-2271-4f3e-9e8d-bde405643edb admin admin] RBAC: Authorizing `identity:delete_user(user_id=300105986ab64ca69583b2a4305ad6c8)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:19.674896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a689e-2271-4f3e-9e8d-bde405643edb admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:19.749516 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:19.808435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-625a689e-2271-4f3e-9e8d-bde405643edb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '300105986ab64ca69583b2a4305ad6c8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:19.821096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-625a689e-2271-4f3e-9e8d-bde405643edb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '300105986ab64ca69583b2a4305ad6c8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:19.896423 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-625a689e-2271-4f3e-9e8d-bde405643edb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '300105986ab64ca69583b2a4305ad6c8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:19.896545 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-625a689e-2271-4f3e-9e8d-bde405643edb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '300105986ab64ca69583b2a4305ad6c8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:19.897146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-625a689e-2271-4f3e-9e8d-bde405643edb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '300105986ab64ca69583b2a4305ad6c8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:19.909925 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1294/2600] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:19 2022] DELETE /identity/v3/users/300105986ab64ca69583b2a4305ad6c8 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:19.914603 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb112b-ea4d-468d-bc19-dd4062eda5d2 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:19.915220 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb112b-ea4d-468d-bc19-dd4062eda5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:19.915220 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb112b-ea4d-468d-bc19-dd4062eda5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:19.933739 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-63fb112b-ea4d-468d-bc19-dd4062eda5d2 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:19.935027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-63fb112b-ea4d-468d-bc19-dd4062eda5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:19.977317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:19.995041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1307/2601] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:20.009016 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a689e-2271-4f3e-9e8d-bde405643edb admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:20.009016 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-625a689e-2271-4f3e-9e8d-bde405643edb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:20.065614 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a689e-2271-4f3e-9e8d-bde405643edb admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:20.069380 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4874ff-a068-4452-8a26-f17ece0c5dec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:20.070030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4874ff-a068-4452-8a26-f17ece0c5dec service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:20.070318 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4874ff-a068-4452-8a26-f17ece0c5dec service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:20.070630 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4874ff-a068-4452-8a26-f17ece0c5dec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:20.079621 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4874ff-a068-4452-8a26-f17ece0c5dec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:20.081092 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4874ff-a068-4452-8a26-f17ece0c5dec service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:20.090695 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1295/2602] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:20.107954 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:20.493440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb112b-ea4d-468d-bc19-dd4062eda5d2 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:20.494939 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-63fb112b-ea4d-468d-bc19-dd4062eda5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:20.508761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb112b-ea4d-468d-bc19-dd4062eda5d2 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:20.514773 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4370b60-25e8-499e-91ea-15e610eca436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:20.515653 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4370b60-25e8-499e-91ea-15e610eca436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:20.515942 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4370b60-25e8-499e-91ea-15e610eca436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:20.516257 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4370b60-25e8-499e-91ea-15e610eca436 admin admin] PATH_INFO: `/v3/projects/c1a26c3444f240798c013d1b72d58794` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:20.517468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4370b60-25e8-499e-91ea-15e610eca436 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1a26c3444f240798c013d1b72d58794)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:20.519450 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4370b60-25e8-499e-91ea-15e610eca436 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:20.622293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a4370b60-25e8-499e-91ea-15e610eca436 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1a26c3444f240798c013d1b72d58794', 'request_id': 'req-a4370b60-25e8-499e-91ea-15e610eca436'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:20.628469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a4370b60-25e8-499e-91ea-15e610eca436 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1a26c3444f240798c013d1b72d58794', 'request_id': 'req-a4370b60-25e8-499e-91ea-15e610eca436'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:20.634819 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a4370b60-25e8-499e-91ea-15e610eca436 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1a26c3444f240798c013d1b72d58794', 'request_id': 'req-a4370b60-25e8-499e-91ea-15e610eca436'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:20.695504 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a4370b60-25e8-499e-91ea-15e610eca436 admin admin] The token cache is being invalidate because project c1a26c3444f240798c013d1b72d58794 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:20.695821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a4370b60-25e8-499e-91ea-15e610eca436 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:20.696447 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a4370b60-25e8-499e-91ea-15e610eca436 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:20.698024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1308/2603] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:20 2022] DELETE /identity/v3/projects/c1a26c3444f240798c013d1b72d58794 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:20.701980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4874ff-a068-4452-8a26-f17ece0c5dec service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:20.712528 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4874ff-a068-4452-8a26-f17ece0c5dec service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:20.775256 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56530dbb-9412-43ba-9f5d-19457dd4cc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:20.776130 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56530dbb-9412-43ba-9f5d-19457dd4cc44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:20.776411 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56530dbb-9412-43ba-9f5d-19457dd4cc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:20.776706 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56530dbb-9412-43ba-9f5d-19457dd4cc44 admin admin] PATH_INFO: `/v3/users/fb653efd630245c39cba81bdda223b18` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:20.777877 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56530dbb-9412-43ba-9f5d-19457dd4cc44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb653efd630245c39cba81bdda223b18)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:20.779816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56530dbb-9412-43ba-9f5d-19457dd4cc44 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:20.782110 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:20.974632 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-56530dbb-9412-43ba-9f5d-19457dd4cc44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb653efd630245c39cba81bdda223b18'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:20.986824 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-56530dbb-9412-43ba-9f5d-19457dd4cc44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb653efd630245c39cba81bdda223b18'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:21.092518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-56530dbb-9412-43ba-9f5d-19457dd4cc44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb653efd630245c39cba81bdda223b18'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:21.093232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-56530dbb-9412-43ba-9f5d-19457dd4cc44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb653efd630245c39cba81bdda223b18'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:21.093836 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-56530dbb-9412-43ba-9f5d-19457dd4cc44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb653efd630245c39cba81bdda223b18'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:21.107818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1296/2604] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:20 2022] DELETE /identity/v3/users/fb653efd630245c39cba81bdda223b18 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:21.548762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4370b60-25e8-499e-91ea-15e610eca436 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:21.550379 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a4370b60-25e8-499e-91ea-15e610eca436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:21.564530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4370b60-25e8-499e-91ea-15e610eca436 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:21.624861 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79116d3b-99b6-4533-9354-c28819454df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:21.625720 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79116d3b-99b6-4533-9354-c28819454df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:21.626002 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79116d3b-99b6-4533-9354-c28819454df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:21.626280 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79116d3b-99b6-4533-9354-c28819454df1 admin admin] PATH_INFO: `/v3/projects/0513b9e88c5e4d69bceb5e9d1d5b8a37` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:21.627492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79116d3b-99b6-4533-9354-c28819454df1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0513b9e88c5e4d69bceb5e9d1d5b8a37)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:21.629393 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79116d3b-99b6-4533-9354-c28819454df1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:21.737728 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-79116d3b-99b6-4533-9354-c28819454df1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0513b9e88c5e4d69bceb5e9d1d5b8a37', 'request_id': 'req-79116d3b-99b6-4533-9354-c28819454df1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:21.745440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-79116d3b-99b6-4533-9354-c28819454df1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0513b9e88c5e4d69bceb5e9d1d5b8a37', 'request_id': 'req-79116d3b-99b6-4533-9354-c28819454df1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:21.750943 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-79116d3b-99b6-4533-9354-c28819454df1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0513b9e88c5e4d69bceb5e9d1d5b8a37', 'request_id': 'req-79116d3b-99b6-4533-9354-c28819454df1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:21.841348 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:21.847442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-79116d3b-99b6-4533-9354-c28819454df1 admin admin] The token cache is being invalidate because project 0513b9e88c5e4d69bceb5e9d1d5b8a37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:21.847762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-79116d3b-99b6-4533-9354-c28819454df1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:21.848461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-79116d3b-99b6-4533-9354-c28819454df1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:21.850054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1309/2605] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:21 2022] DELETE /identity/v3/projects/0513b9e88c5e4d69bceb5e9d1d5b8a37 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:21.856728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc50733-15b2-4fec-b067-c3766763c548 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:21.857192 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc50733-15b2-4fec-b067-c3766763c548 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:21.857690 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc50733-15b2-4fec-b067-c3766763c548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:21.882397 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-bcc50733-15b2-4fec-b067-c3766763c548 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:21.884187 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-bcc50733-15b2-4fec-b067-c3766763c548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:21.947274 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1297/2606] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:21.954970 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3cd8c-c515-4539-a813-0be0532feeb6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:21.955270 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3cd8c-c515-4539-a813-0be0532feeb6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:21.955558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3cd8c-c515-4539-a813-0be0532feeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:21.974105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-c7e3cd8c-c515-4539-a813-0be0532feeb6 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:21.975361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c7e3cd8c-c515-4539-a813-0be0532feeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:21.983886 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1310/2607] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:21.988012 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc50733-15b2-4fec-b067-c3766763c548 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:21.998390 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc50733-15b2-4fec-b067-c3766763c548 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:22.003035 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92044b-3d1a-42ac-8881-bc45b5210a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:22.003687 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92044b-3d1a-42ac-8881-bc45b5210a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:22.003975 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92044b-3d1a-42ac-8881-bc45b5210a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:22.004255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92044b-3d1a-42ac-8881-bc45b5210a1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:22.004979 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92044b-3d1a-42ac-8881-bc45b5210a1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:25:22.005290 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92044b-3d1a-42ac-8881-bc45b5210a1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:22.006922 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92044b-3d1a-42ac-8881-bc45b5210a1a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:22.018758 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1298/2608] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:25:22.022963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3cd8c-c515-4539-a813-0be0532feeb6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:22.032936 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3cd8c-c515-4539-a813-0be0532feeb6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:22.037592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc26062-7a47-404a-9cca-07491bad7a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:22.038256 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc26062-7a47-404a-9cca-07491bad7a22 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:22.038600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc26062-7a47-404a-9cca-07491bad7a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:22.039424 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc26062-7a47-404a-9cca-07491bad7a22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:22.040133 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc26062-7a47-404a-9cca-07491bad7a22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:22.042056 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc26062-7a47-404a-9cca-07491bad7a22 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:22.157167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1311/2609] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:22 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:22.161511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92044b-3d1a-42ac-8881-bc45b5210a1a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:22.173092 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92044b-3d1a-42ac-8881-bc45b5210a1a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:22.231481 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec23b3-27e6-4a28-bf75-bf53fb69910f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:22.232157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec23b3-27e6-4a28-bf75-bf53fb69910f admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:22.232447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec23b3-27e6-4a28-bf75-bf53fb69910f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:22.232730 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ec23b3-27e6-4a28-bf75-bf53fb69910f admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:22.233466 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec23b3-27e6-4a28-bf75-bf53fb69910f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:22.235436 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ec23b3-27e6-4a28-bf75-bf53fb69910f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:22.314881 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:22.321378 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1299/2610] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:22 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:22.325558 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc26062-7a47-404a-9cca-07491bad7a22 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:22.336446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc26062-7a47-404a-9cca-07491bad7a22 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:22.341183 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28145a-2146-43e7-911c-c138165e45e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:22.341841 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28145a-2146-43e7-911c-c138165e45e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:22.342131 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28145a-2146-43e7-911c-c138165e45e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:22.342446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28145a-2146-43e7-911c-c138165e45e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:22.343271 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28145a-2146-43e7-911c-c138165e45e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:22.344882 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28145a-2146-43e7-911c-c138165e45e4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:22.355282 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1312/2611] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:22.359448 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec23b3-27e6-4a28-bf75-bf53fb69910f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:22.369385 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ec23b3-27e6-4a28-bf75-bf53fb69910f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:22.374760 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f81b7-7d2a-4eca-92c9-7145d26997be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:22.375356 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f81b7-7d2a-4eca-92c9-7145d26997be admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:22.378625 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f81b7-7d2a-4eca-92c9-7145d26997be admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:22.378625 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f81b7-7d2a-4eca-92c9-7145d26997be admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:22.378625 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f81b7-7d2a-4eca-92c9-7145d26997be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:22.378625 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f81b7-7d2a-4eca-92c9-7145d26997be admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:22.389785 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1300/2612] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:22.393708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28145a-2146-43e7-911c-c138165e45e4 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:22.405459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28145a-2146-43e7-911c-c138165e45e4 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:22.410188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71461113-d582-4046-93ed-4dd6f2edd1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:22.411349 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71461113-d582-4046-93ed-4dd6f2edd1fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:22.411793 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71461113-d582-4046-93ed-4dd6f2edd1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:22.412082 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71461113-d582-4046-93ed-4dd6f2edd1fc admin admin] PATH_INFO: `/v3/projects/a3cee9e2e5b84b2193fe5c41081feafa/users/210286e3dd1f4480a332fdafa531c5fd/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:22.427619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71461113-d582-4046-93ed-4dd6f2edd1fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3cee9e2e5b84b2193fe5c41081feafa, user_id=210286e3dd1f4480a332fdafa531c5fd, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:22.430354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71461113-d582-4046-93ed-4dd6f2edd1fc admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:22.543217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1313/2613] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:22 2022] PUT /identity/v3/projects/a3cee9e2e5b84b2193fe5c41081feafa/users/210286e3dd1f4480a332fdafa531c5fd/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:22.554569 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2187e45-86f5-4108-a69b-534662427c22 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:22.555364 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2187e45-86f5-4108-a69b-534662427c22 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:22.555727 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2187e45-86f5-4108-a69b-534662427c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:22.583177 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e2187e45-86f5-4108-a69b-534662427c22 None None] MFA Rules not processed for user `210286e3dd1f4480a332fdafa531c5fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:22.584427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e2187e45-86f5-4108-a69b-534662427c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:22.653712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1301/2614] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:22.658310 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71461113-d582-4046-93ed-4dd6f2edd1fc admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:22.724265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71461113-d582-4046-93ed-4dd6f2edd1fc admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:22.730075 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d94f1d-f4ba-4487-83ff-e1063748534d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:22.730912 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d94f1d-f4ba-4487-83ff-e1063748534d admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:22.731377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d94f1d-f4ba-4487-83ff-e1063748534d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:22.731811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d94f1d-f4ba-4487-83ff-e1063748534d admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:22.732569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d94f1d-f4ba-4487-83ff-e1063748534d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:22.734587 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d94f1d-f4ba-4487-83ff-e1063748534d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:22.808703 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1314/2615] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:22 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:22.813832 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2187e45-86f5-4108-a69b-534662427c22 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:22.826176 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2187e45-86f5-4108-a69b-534662427c22 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:22.885205 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677427e-1f94-4217-b287-4c9d376b868c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:22.886910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677427e-1f94-4217-b287-4c9d376b868c admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:22.886910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677427e-1f94-4217-b287-4c9d376b868c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:22.886910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677427e-1f94-4217-b287-4c9d376b868c admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:22.891527 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b677427e-1f94-4217-b287-4c9d376b868c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:22.891527 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b677427e-1f94-4217-b287-4c9d376b868c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:22.959616 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1302/2616] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:22 2022] POST /identity/v3/users => generated 296 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:22.963687 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d94f1d-f4ba-4487-83ff-e1063748534d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:22.974255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d94f1d-f4ba-4487-83ff-e1063748534d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:22.979600 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683beb73-a401-4900-b8cd-3a2170821f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:22.980262 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683beb73-a401-4900-b8cd-3a2170821f50 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:22.980548 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683beb73-a401-4900-b8cd-3a2170821f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:22.980831 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683beb73-a401-4900-b8cd-3a2170821f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:22.981567 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683beb73-a401-4900-b8cd-3a2170821f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:22.983214 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683beb73-a401-4900-b8cd-3a2170821f50 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:22.994239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1315/2617] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:22.998552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677427e-1f94-4217-b287-4c9d376b868c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:23.008566 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677427e-1f94-4217-b287-4c9d376b868c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:23.013216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a08a7-00a3-4d3d-a5ae-69705f9b7910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:23.014101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a08a7-00a3-4d3d-a5ae-69705f9b7910 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:23.014416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a08a7-00a3-4d3d-a5ae-69705f9b7910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:23.014703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a08a7-00a3-4d3d-a5ae-69705f9b7910 admin admin] PATH_INFO: `/v3/projects/8d245f25d19540e4832a72e63ca39b07/users/749b3f9afe1b4707a3a5f127c8a782c8/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:23.032216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a08a7-00a3-4d3d-a5ae-69705f9b7910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d245f25d19540e4832a72e63ca39b07, user_id=749b3f9afe1b4707a3a5f127c8a782c8, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:23.035337 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a08a7-00a3-4d3d-a5ae-69705f9b7910 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:23.154409 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1303/2618] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:22 2022] PUT /identity/v3/projects/8d245f25d19540e4832a72e63ca39b07/users/749b3f9afe1b4707a3a5f127c8a782c8/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:23.159640 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683beb73-a401-4900-b8cd-3a2170821f50 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:23.170193 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683beb73-a401-4900-b8cd-3a2170821f50 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:23.228190 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce357e-2436-4b70-8b8f-0de6cab2d9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:23.228873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce357e-2436-4b70-8b8f-0de6cab2d9c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:23.229162 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce357e-2436-4b70-8b8f-0de6cab2d9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:23.229445 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce357e-2436-4b70-8b8f-0de6cab2d9c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:23.230179 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce357e-2436-4b70-8b8f-0de6cab2d9c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:23.231841 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce357e-2436-4b70-8b8f-0de6cab2d9c9 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:23.241612 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1316/2619] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:23.246637 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a08a7-00a3-4d3d-a5ae-69705f9b7910 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:23.256949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a08a7-00a3-4d3d-a5ae-69705f9b7910 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:23.261582 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325ddf7-8b59-4b3b-a5ca-67964319387a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:23.262506 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325ddf7-8b59-4b3b-a5ca-67964319387a admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:23.262794 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325ddf7-8b59-4b3b-a5ca-67964319387a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:23.263074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325ddf7-8b59-4b3b-a5ca-67964319387a admin admin] PATH_INFO: `/v3/projects/8d245f25d19540e4832a72e63ca39b07/users/749b3f9afe1b4707a3a5f127c8a782c8/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:23.265036 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325ddf7-8b59-4b3b-a5ca-67964319387a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d245f25d19540e4832a72e63ca39b07, user_id=749b3f9afe1b4707a3a5f127c8a782c8, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:23.267716 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6325ddf7-8b59-4b3b-a5ca-67964319387a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:23.278969 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1304/2620] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:23 2022] PUT /identity/v3/projects/8d245f25d19540e4832a72e63ca39b07/users/749b3f9afe1b4707a3a5f127c8a782c8/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:23.289046 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e62e3-8363-43aa-a49f-73cc56d08b0b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:23.290064 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e62e3-8363-43aa-a49f-73cc56d08b0b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:23.290460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e62e3-8363-43aa-a49f-73cc56d08b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:23.307176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:23.321872 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4e7e62e3-8363-43aa-a49f-73cc56d08b0b None None] MFA Rules not processed for user `749b3f9afe1b4707a3a5f127c8a782c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:23.323141 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4e7e62e3-8363-43aa-a49f-73cc56d08b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:23.393972 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1317/2621] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:23.406363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325ddf7-8b59-4b3b-a5ca-67964319387a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:23.408031 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-6325ddf7-8b59-4b3b-a5ca-67964319387a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:23.439919 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:23.467368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325ddf7-8b59-4b3b-a5ca-67964319387a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:23.470972 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963af432-12ef-4729-906a-a2b61f1d63ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:23.471676 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963af432-12ef-4729-906a-a2b61f1d63ff service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:23.471962 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963af432-12ef-4729-906a-a2b61f1d63ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:23.472236 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963af432-12ef-4729-906a-a2b61f1d63ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:23.482386 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963af432-12ef-4729-906a-a2b61f1d63ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:23.483865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963af432-12ef-4729-906a-a2b61f1d63ff service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:23.493950 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1305/2622] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:23.854443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e62e3-8363-43aa-a49f-73cc56d08b0b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:23.855908 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4e7e62e3-8363-43aa-a49f-73cc56d08b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:23.932540 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963af432-12ef-4729-906a-a2b61f1d63ff service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:23.933928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-963af432-12ef-4729-906a-a2b61f1d63ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:23.935278 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e62e3-8363-43aa-a49f-73cc56d08b0b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:23.940259 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1ce79-c40a-4f02-94d6-bd5a8f0ff6fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7662e61619d44c92af3ffacbacba3913', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:23.940983 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1ce79-c40a-4f02-94d6-bd5a8f0ff6fb service placement] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:23.941270 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1ce79-c40a-4f02-94d6-bd5a8f0ff6fb service placement] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:23.941547 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1ce79-c40a-4f02-94d6-bd5a8f0ff6fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:23.943315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-bfb1ce79-c40a-4f02-94d6-bd5a8f0ff6fb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:23.956988 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1ce79-c40a-4f02-94d6-bd5a8f0ff6fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:23.961892 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1ce79-c40a-4f02-94d6-bd5a8f0ff6fb service placement] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:23.970447 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1318/2623] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:24.004333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1ce79-c40a-4f02-94d6-bd5a8f0ff6fb service placement] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:24.005257 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963af432-12ef-4729-906a-a2b61f1d63ff service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:24.010947 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489de8f2-cc17-4e45-bdb7-b3fd32cccaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:24.013196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489de8f2-cc17-4e45-bdb7-b3fd32cccaa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:24.013196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489de8f2-cc17-4e45-bdb7-b3fd32cccaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:24.013196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489de8f2-cc17-4e45-bdb7-b3fd32cccaa5 admin admin] PATH_INFO: `/v3/users/a6ce7b9ff5da4e9eae1b7615eafb82aa` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:24.013628 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489de8f2-cc17-4e45-bdb7-b3fd32cccaa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6ce7b9ff5da4e9eae1b7615eafb82aa)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:24.015601 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489de8f2-cc17-4e45-bdb7-b3fd32cccaa5 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:24.016627 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1ce79-c40a-4f02-94d6-bd5a8f0ff6fb service placement] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:24.021461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a752d6f-5b86-4f6e-bb58-49677c87ed1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:24.022339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a752d6f-5b86-4f6e-bb58-49677c87ed1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:24.022793 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a752d6f-5b86-4f6e-bb58-49677c87ed1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:24.024505 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a752d6f-5b86-4f6e-bb58-49677c87ed1e admin admin] PATH_INFO: `/v3/users/210286e3dd1f4480a332fdafa531c5fd` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:24.025448 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a752d6f-5b86-4f6e-bb58-49677c87ed1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=210286e3dd1f4480a332fdafa531c5fd)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:24.027433 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a752d6f-5b86-4f6e-bb58-49677c87ed1e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:24.268985 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-489de8f2-cc17-4e45-bdb7-b3fd32cccaa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6ce7b9ff5da4e9eae1b7615eafb82aa'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:24.278074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2a752d6f-5b86-4f6e-bb58-49677c87ed1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '210286e3dd1f4480a332fdafa531c5fd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:24.283818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-489de8f2-cc17-4e45-bdb7-b3fd32cccaa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6ce7b9ff5da4e9eae1b7615eafb82aa'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:24.294086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2a752d6f-5b86-4f6e-bb58-49677c87ed1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '210286e3dd1f4480a332fdafa531c5fd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:24.294856 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2a752d6f-5b86-4f6e-bb58-49677c87ed1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '210286e3dd1f4480a332fdafa531c5fd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:24.295488 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2a752d6f-5b86-4f6e-bb58-49677c87ed1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '210286e3dd1f4480a332fdafa531c5fd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:24.303526 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-2a752d6f-5b86-4f6e-bb58-49677c87ed1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '210286e3dd1f4480a332fdafa531c5fd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:24.306847 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:24.366155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-489de8f2-cc17-4e45-bdb7-b3fd32cccaa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6ce7b9ff5da4e9eae1b7615eafb82aa'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:24.366799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-489de8f2-cc17-4e45-bdb7-b3fd32cccaa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6ce7b9ff5da4e9eae1b7615eafb82aa'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:24.368006 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-489de8f2-cc17-4e45-bdb7-b3fd32cccaa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6ce7b9ff5da4e9eae1b7615eafb82aa'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:24.381336 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1306/2624] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:23 2022] DELETE /identity/v3/users/a6ce7b9ff5da4e9eae1b7615eafb82aa => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:24.387337 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8187de-57a1-4623-8ad9-439dc4df615f None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:24.387629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8187de-57a1-4623-8ad9-439dc4df615f None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:24.387913 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8187de-57a1-4623-8ad9-439dc4df615f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:24.405387 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-0e8187de-57a1-4623-8ad9-439dc4df615f None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:24.406627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0e8187de-57a1-4623-8ad9-439dc4df615f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:24.429384 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1319/2625] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:24 2022] DELETE /identity/v3/users/210286e3dd1f4480a332fdafa531c5fd => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:24.439317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced17c0-33d6-4545-9960-dee58323c41b None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:24.439740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced17c0-33d6-4545-9960-dee58323c41b None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:24.440187 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced17c0-33d6-4545-9960-dee58323c41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:24.457468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:24.459564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5ced17c0-33d6-4545-9960-dee58323c41b None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:24.460794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5ced17c0-33d6-4545-9960-dee58323c41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:24.471822 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1307/2626] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:24.482922 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8187de-57a1-4623-8ad9-439dc4df615f None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:24.484324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-0e8187de-57a1-4623-8ad9-439dc4df615f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:24.521079 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1320/2627] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:24.531492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced17c0-33d6-4545-9960-dee58323c41b None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:24.532836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5ced17c0-33d6-4545-9960-dee58323c41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:24.543097 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8187de-57a1-4623-8ad9-439dc4df615f None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:24.594889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced17c0-33d6-4545-9960-dee58323c41b None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:24.600233 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d9117-406f-4575-b79e-a0bc0f06b143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:24.601173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d9117-406f-4575-b79e-a0bc0f06b143 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:24.602321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d9117-406f-4575-b79e-a0bc0f06b143 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:24.605872 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d9117-406f-4575-b79e-a0bc0f06b143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:24.609674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c138251-665f-4b6a-8f71-6a3bb9c7215c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:24.610325 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c138251-665f-4b6a-8f71-6a3bb9c7215c service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:24.610633 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c138251-665f-4b6a-8f71-6a3bb9c7215c service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:24.610917 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c138251-665f-4b6a-8f71-6a3bb9c7215c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:24.614413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d9117-406f-4575-b79e-a0bc0f06b143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:24.615891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d9117-406f-4575-b79e-a0bc0f06b143 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:24.620587 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c138251-665f-4b6a-8f71-6a3bb9c7215c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:24.622045 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c138251-665f-4b6a-8f71-6a3bb9c7215c service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:24.626110 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1321/2628] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:25:24.632723 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1308/2629] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:25:25.041561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d9117-406f-4575-b79e-a0bc0f06b143 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:25.046611 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-116d9117-406f-4575-b79e-a0bc0f06b143 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:25.063320 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d9117-406f-4575-b79e-a0bc0f06b143 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:25.068068 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e5bbd-55a7-4236-9a29-f9da10fae0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:25.069166 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e5bbd-55a7-4236-9a29-f9da10fae0e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:25.069467 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c138251-665f-4b6a-8f71-6a3bb9c7215c service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:25.069908 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e5bbd-55a7-4236-9a29-f9da10fae0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:25.071388 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e5bbd-55a7-4236-9a29-f9da10fae0e2 admin admin] PATH_INFO: `/v3/projects/c8495d512a9a47368dbb2b585b385b71` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:25.072142 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1c138251-665f-4b6a-8f71-6a3bb9c7215c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:25.075225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e5bbd-55a7-4236-9a29-f9da10fae0e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8495d512a9a47368dbb2b585b385b71)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:25.075225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e5bbd-55a7-4236-9a29-f9da10fae0e2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:25.089959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c138251-665f-4b6a-8f71-6a3bb9c7215c service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:25.095710 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160390b-3e02-4d02-a9e9-f3a3c5d93b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:25.096688 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160390b-3e02-4d02-a9e9-f3a3c5d93b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:25.097068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160390b-3e02-4d02-a9e9-f3a3c5d93b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:25.097433 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160390b-3e02-4d02-a9e9-f3a3c5d93b58 admin admin] PATH_INFO: `/v3/projects/a3cee9e2e5b84b2193fe5c41081feafa` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:25.099102 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160390b-3e02-4d02-a9e9-f3a3c5d93b58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3cee9e2e5b84b2193fe5c41081feafa)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:25.102463 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160390b-3e02-4d02-a9e9-f3a3c5d93b58 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:25.176767 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-900e5bbd-55a7-4236-9a29-f9da10fae0e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8495d512a9a47368dbb2b585b385b71', 'request_id': 'req-900e5bbd-55a7-4236-9a29-f9da10fae0e2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.183203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-900e5bbd-55a7-4236-9a29-f9da10fae0e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8495d512a9a47368dbb2b585b385b71', 'request_id': 'req-900e5bbd-55a7-4236-9a29-f9da10fae0e2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.189177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-900e5bbd-55a7-4236-9a29-f9da10fae0e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8495d512a9a47368dbb2b585b385b71', 'request_id': 'req-900e5bbd-55a7-4236-9a29-f9da10fae0e2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.206970 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3160390b-3e02-4d02-a9e9-f3a3c5d93b58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3cee9e2e5b84b2193fe5c41081feafa', 'request_id': 'req-3160390b-3e02-4d02-a9e9-f3a3c5d93b58'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.275532 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-900e5bbd-55a7-4236-9a29-f9da10fae0e2 admin admin] The token cache is being invalidate because project c8495d512a9a47368dbb2b585b385b71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:25.275733 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-900e5bbd-55a7-4236-9a29-f9da10fae0e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.276376 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-900e5bbd-55a7-4236-9a29-f9da10fae0e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.277931 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1322/2630] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:25 2022] DELETE /identity/v3/projects/c8495d512a9a47368dbb2b585b385b71 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:25.281982 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e5bbd-55a7-4236-9a29-f9da10fae0e2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:25.283334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-900e5bbd-55a7-4236-9a29-f9da10fae0e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:25.305546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:25.349043 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e5bbd-55a7-4236-9a29-f9da10fae0e2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:25.349281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3160390b-3e02-4d02-a9e9-f3a3c5d93b58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3cee9e2e5b84b2193fe5c41081feafa', 'request_id': 'req-3160390b-3e02-4d02-a9e9-f3a3c5d93b58'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.355571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3160390b-3e02-4d02-a9e9-f3a3c5d93b58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3cee9e2e5b84b2193fe5c41081feafa', 'request_id': 'req-3160390b-3e02-4d02-a9e9-f3a3c5d93b58'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.367008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3160390b-3e02-4d02-a9e9-f3a3c5d93b58 admin admin] The token cache is being invalidate because project a3cee9e2e5b84b2193fe5c41081feafa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:25.367352 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3160390b-3e02-4d02-a9e9-f3a3c5d93b58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.368373 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3160390b-3e02-4d02-a9e9-f3a3c5d93b58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.369908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1309/2631] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:25 2022] DELETE /identity/v3/projects/a3cee9e2e5b84b2193fe5c41081feafa => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:25.373888 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160390b-3e02-4d02-a9e9-f3a3c5d93b58 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:25.375270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3160390b-3e02-4d02-a9e9-f3a3c5d93b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:25.410577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c6f63-a4ee-48b0-9e52-221caa0baa46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:25.411490 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c6f63-a4ee-48b0-9e52-221caa0baa46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:25.411776 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c6f63-a4ee-48b0-9e52-221caa0baa46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:25.412053 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c6f63-a4ee-48b0-9e52-221caa0baa46 admin admin] PATH_INFO: `/v3/users/d9dd8b6cea1d46fea1543786ab978372` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:25.412665 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:25.413284 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c6f63-a4ee-48b0-9e52-221caa0baa46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9dd8b6cea1d46fea1543786ab978372)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:25.415323 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c6f63-a4ee-48b0-9e52-221caa0baa46 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:25.441641 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160390b-3e02-4d02-a9e9-f3a3c5d93b58 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:25.447396 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9897f-95ad-440f-8bcd-d008915cec1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:25.448273 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9897f-95ad-440f-8bcd-d008915cec1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:25.448558 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9897f-95ad-440f-8bcd-d008915cec1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:25.448838 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9897f-95ad-440f-8bcd-d008915cec1d admin admin] PATH_INFO: `/v3/users/749b3f9afe1b4707a3a5f127c8a782c8` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:25.450128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9897f-95ad-440f-8bcd-d008915cec1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=749b3f9afe1b4707a3a5f127c8a782c8)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:25.452094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9897f-95ad-440f-8bcd-d008915cec1d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:25.632518 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d92c6f63-a4ee-48b0-9e52-221caa0baa46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9dd8b6cea1d46fea1543786ab978372'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.645030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d92c6f63-a4ee-48b0-9e52-221caa0baa46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9dd8b6cea1d46fea1543786ab978372'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.645700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d92c6f63-a4ee-48b0-9e52-221caa0baa46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9dd8b6cea1d46fea1543786ab978372'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.646317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d92c6f63-a4ee-48b0-9e52-221caa0baa46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9dd8b6cea1d46fea1543786ab978372'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.653644 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-d92c6f63-a4ee-48b0-9e52-221caa0baa46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9dd8b6cea1d46fea1543786ab978372'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.688374 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-91d9897f-95ad-440f-8bcd-d008915cec1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '749b3f9afe1b4707a3a5f127c8a782c8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.700671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-91d9897f-95ad-440f-8bcd-d008915cec1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '749b3f9afe1b4707a3a5f127c8a782c8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.765705 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1323/2632] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:25 2022] DELETE /identity/v3/users/d9dd8b6cea1d46fea1543786ab978372 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:25.846270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-91d9897f-95ad-440f-8bcd-d008915cec1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '749b3f9afe1b4707a3a5f127c8a782c8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.847088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-91d9897f-95ad-440f-8bcd-d008915cec1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '749b3f9afe1b4707a3a5f127c8a782c8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.847700 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-91d9897f-95ad-440f-8bcd-d008915cec1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '749b3f9afe1b4707a3a5f127c8a782c8'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:25.868962 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1310/2633] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:25 2022] DELETE /identity/v3/users/749b3f9afe1b4707a3a5f127c8a782c8 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:26.323067 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c6f63-a4ee-48b0-9e52-221caa0baa46 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:26.324715 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d92c6f63-a4ee-48b0-9e52-221caa0baa46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:26.376950 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:26.393103 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c6f63-a4ee-48b0-9e52-221caa0baa46 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:26.413603 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9897f-95ad-440f-8bcd-d008915cec1d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:26.415097 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-91d9897f-95ad-440f-8bcd-d008915cec1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:26.464334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f85cf4-64b4-40f9-a8e1-0cbb823ee3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:26.465435 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f85cf4-64b4-40f9-a8e1-0cbb823ee3b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:26.465723 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f85cf4-64b4-40f9-a8e1-0cbb823ee3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:26.466295 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f85cf4-64b4-40f9-a8e1-0cbb823ee3b3 admin admin] PATH_INFO: `/v3/projects/6005f6f09c8c4fc99e7d23f795373265` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:26.467355 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f85cf4-64b4-40f9-a8e1-0cbb823ee3b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6005f6f09c8c4fc99e7d23f795373265)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:26.469353 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f85cf4-64b4-40f9-a8e1-0cbb823ee3b3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:26.497305 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9897f-95ad-440f-8bcd-d008915cec1d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:26.502177 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95c5e5-32c8-4efb-9ea2-f9d913dfb412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:26.503074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95c5e5-32c8-4efb-9ea2-f9d913dfb412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:26.503361 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95c5e5-32c8-4efb-9ea2-f9d913dfb412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:26.503640 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95c5e5-32c8-4efb-9ea2-f9d913dfb412 admin admin] PATH_INFO: `/v3/projects/8d245f25d19540e4832a72e63ca39b07` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:26.504831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95c5e5-32c8-4efb-9ea2-f9d913dfb412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d245f25d19540e4832a72e63ca39b07)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:26.506887 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95c5e5-32c8-4efb-9ea2-f9d913dfb412 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:26.607712 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b2f85cf4-64b4-40f9-a8e1-0cbb823ee3b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6005f6f09c8c4fc99e7d23f795373265', 'request_id': 'req-b2f85cf4-64b4-40f9-a8e1-0cbb823ee3b3'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:26.608527 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0e95c5e5-32c8-4efb-9ea2-f9d913dfb412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d245f25d19540e4832a72e63ca39b07', 'request_id': 'req-0e95c5e5-32c8-4efb-9ea2-f9d913dfb412'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:26.612126 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:26.614650 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b2f85cf4-64b4-40f9-a8e1-0cbb823ee3b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6005f6f09c8c4fc99e7d23f795373265', 'request_id': 'req-b2f85cf4-64b4-40f9-a8e1-0cbb823ee3b3'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:26.620872 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b2f85cf4-64b4-40f9-a8e1-0cbb823ee3b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6005f6f09c8c4fc99e7d23f795373265', 'request_id': 'req-b2f85cf4-64b4-40f9-a8e1-0cbb823ee3b3'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:26.674851 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0e95c5e5-32c8-4efb-9ea2-f9d913dfb412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d245f25d19540e4832a72e63ca39b07', 'request_id': 'req-0e95c5e5-32c8-4efb-9ea2-f9d913dfb412'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:26.681892 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0e95c5e5-32c8-4efb-9ea2-f9d913dfb412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d245f25d19540e4832a72e63ca39b07', 'request_id': 'req-0e95c5e5-32c8-4efb-9ea2-f9d913dfb412'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:26.694094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0e95c5e5-32c8-4efb-9ea2-f9d913dfb412 admin admin] The token cache is being invalidate because project 8d245f25d19540e4832a72e63ca39b07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:26.694396 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0e95c5e5-32c8-4efb-9ea2-f9d913dfb412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:26.695152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-0e95c5e5-32c8-4efb-9ea2-f9d913dfb412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:26.696905 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1311/2634] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:26 2022] DELETE /identity/v3/projects/8d245f25d19540e4832a72e63ca39b07 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:26.701650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249727b-0a3e-4c29-bc29-a1a7afd678cd None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:26.701948 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249727b-0a3e-4c29-bc29-a1a7afd678cd None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:26.702235 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249727b-0a3e-4c29-bc29-a1a7afd678cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:26.720004 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9249727b-0a3e-4c29-bc29-a1a7afd678cd None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:26.721207 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9249727b-0a3e-4c29-bc29-a1a7afd678cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:26.765394 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b2f85cf4-64b4-40f9-a8e1-0cbb823ee3b3 admin admin] The token cache is being invalidate because project 6005f6f09c8c4fc99e7d23f795373265 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:26.765730 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b2f85cf4-64b4-40f9-a8e1-0cbb823ee3b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:26.766483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-b2f85cf4-64b4-40f9-a8e1-0cbb823ee3b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:26.772822 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1324/2635] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:26 2022] DELETE /identity/v3/projects/6005f6f09c8c4fc99e7d23f795373265 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:26.772822 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f85cf4-64b4-40f9-a8e1-0cbb823ee3b3 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:26.773831 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-b2f85cf4-64b4-40f9-a8e1-0cbb823ee3b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:26.781400 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1312/2636] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:26.790896 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013fba1-8f2f-4de6-8e93-e1a54ac56b2a None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:26.791201 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013fba1-8f2f-4de6-8e93-e1a54ac56b2a None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:26.791453 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013fba1-8f2f-4de6-8e93-e1a54ac56b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:26.810182 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4013fba1-8f2f-4de6-8e93-e1a54ac56b2a None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:26.811585 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4013fba1-8f2f-4de6-8e93-e1a54ac56b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:26.819617 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1313/2637] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:26.824123 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013fba1-8f2f-4de6-8e93-e1a54ac56b2a None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:26.833100 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013fba1-8f2f-4de6-8e93-e1a54ac56b2a None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:26.837880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824beab-2242-4766-bb50-22a29db84d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:26.838559 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824beab-2242-4766-bb50-22a29db84d27 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:26.838897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824beab-2242-4766-bb50-22a29db84d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:26.839314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0824beab-2242-4766-bb50-22a29db84d27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:26.839891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824beab-2242-4766-bb50-22a29db84d27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:25:26.840199 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824beab-2242-4766-bb50-22a29db84d27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:26.841815 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0824beab-2242-4766-bb50-22a29db84d27 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:26.846354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f85cf4-64b4-40f9-a8e1-0cbb823ee3b3 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:26.851180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9a8ad-fdee-49af-80f0-cff24780b67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:26.852570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9a8ad-fdee-49af-80f0-cff24780b67f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:26.853467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9a8ad-fdee-49af-80f0-cff24780b67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:26.853762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9a8ad-fdee-49af-80f0-cff24780b67f admin admin] PATH_INFO: `/v3/users/9a473c453c164bb0b0ab3da321510afa` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:26.855006 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1314/2638] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:25:26.855150 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9a8ad-fdee-49af-80f0-cff24780b67f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a473c453c164bb0b0ab3da321510afa)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:26.857057 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e9a8ad-fdee-49af-80f0-cff24780b67f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:26.859276 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824beab-2242-4766-bb50-22a29db84d27 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:26.870066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0824beab-2242-4766-bb50-22a29db84d27 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:26.874833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e5732-3486-4c0d-8b1e-aa9d63070b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:26.875487 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e5732-3486-4c0d-8b1e-aa9d63070b3b admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:26.875773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e5732-3486-4c0d-8b1e-aa9d63070b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:26.876054 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e5732-3486-4c0d-8b1e-aa9d63070b3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:26.876741 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e5732-3486-4c0d-8b1e-aa9d63070b3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:26.878830 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e5732-3486-4c0d-8b1e-aa9d63070b3b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:26.984410 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1315/2639] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:26 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:26.988566 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e5732-3486-4c0d-8b1e-aa9d63070b3b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:27.049403 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-92e9a8ad-fdee-49af-80f0-cff24780b67f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a473c453c164bb0b0ab3da321510afa'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:27.057682 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e5732-3486-4c0d-8b1e-aa9d63070b3b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:27.062564 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83404476-5d8d-4a09-8d0b-3db2bd059ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:27.063091 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-92e9a8ad-fdee-49af-80f0-cff24780b67f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a473c453c164bb0b0ab3da321510afa'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:27.063237 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83404476-5d8d-4a09-8d0b-3db2bd059ccc admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:27.063531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83404476-5d8d-4a09-8d0b-3db2bd059ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:27.063814 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83404476-5d8d-4a09-8d0b-3db2bd059ccc admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:27.064325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-92e9a8ad-fdee-49af-80f0-cff24780b67f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a473c453c164bb0b0ab3da321510afa'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:27.064531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83404476-5d8d-4a09-8d0b-3db2bd059ccc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:27.065054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-92e9a8ad-fdee-49af-80f0-cff24780b67f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a473c453c164bb0b0ab3da321510afa'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:27.066512 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83404476-5d8d-4a09-8d0b-3db2bd059ccc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:27.072366 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-92e9a8ad-fdee-49af-80f0-cff24780b67f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a473c453c164bb0b0ab3da321510afa'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:27.170493 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1325/2640] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:26 2022] DELETE /identity/v3/users/9a473c453c164bb0b0ab3da321510afa => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:27.187098 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1316/2641] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:26 2022] POST /identity/v3/users => generated 287 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:27.188485 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9a8ad-fdee-49af-80f0-cff24780b67f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:27.190046 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-92e9a8ad-fdee-49af-80f0-cff24780b67f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:27.255352 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e9a8ad-fdee-49af-80f0-cff24780b67f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:27.262890 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e7ddb3-26d3-47fb-940c-c13252137848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:27.263562 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7ddb3-26d3-47fb-940c-c13252137848 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:27.263847 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7ddb3-26d3-47fb-940c-c13252137848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:27.264123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7ddb3-26d3-47fb-940c-c13252137848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:27.264850 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e7ddb3-26d3-47fb-940c-c13252137848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:27.266483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e7ddb3-26d3-47fb-940c-c13252137848 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:27.277548 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1326/2642] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:27.281692 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83404476-5d8d-4a09-8d0b-3db2bd059ccc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:27.292413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83404476-5d8d-4a09-8d0b-3db2bd059ccc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:27.297010 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7143750-7cff-4fad-a3f4-9e8afb09cad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:27.297660 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7143750-7cff-4fad-a3f4-9e8afb09cad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:27.297951 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7143750-7cff-4fad-a3f4-9e8afb09cad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:27.298228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7143750-7cff-4fad-a3f4-9e8afb09cad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:27.298984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7143750-7cff-4fad-a3f4-9e8afb09cad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:27.300609 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7143750-7cff-4fad-a3f4-9e8afb09cad9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:27.311265 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1317/2643] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:27.315358 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e7ddb3-26d3-47fb-940c-c13252137848 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:27.326250 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e7ddb3-26d3-47fb-940c-c13252137848 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:27.331531 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcab811-751f-43eb-a370-3b016058563b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:27.332417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcab811-751f-43eb-a370-3b016058563b admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:27.332699 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcab811-751f-43eb-a370-3b016058563b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:27.332972 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcab811-751f-43eb-a370-3b016058563b admin admin] PATH_INFO: `/v3/projects/52e5f9c40d204fb5944f7c004a236f5b/users/2867ea18e4524536a0784181454e681a/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:27.346759 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7143750-7cff-4fad-a3f4-9e8afb09cad9 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:27.348254 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d7143750-7cff-4fad-a3f4-9e8afb09cad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:27.348483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcab811-751f-43eb-a370-3b016058563b admin admin] RBAC: Authorizing `identity:create_grant(project_id=52e5f9c40d204fb5944f7c004a236f5b, user_id=2867ea18e4524536a0784181454e681a, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:27.351199 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcab811-751f-43eb-a370-3b016058563b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:27.361656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7143750-7cff-4fad-a3f4-9e8afb09cad9 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:27.366658 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2c04b-8de3-4675-b038-b8b02e487887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:27.367531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2c04b-8de3-4675-b038-b8b02e487887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:27.369231 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2c04b-8de3-4675-b038-b8b02e487887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:27.369231 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2c04b-8de3-4675-b038-b8b02e487887 admin admin] PATH_INFO: `/v3/users/6a3e633eaf2b4a44bbc653d8f61e9e2d` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:27.370188 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2c04b-8de3-4675-b038-b8b02e487887 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a3e633eaf2b4a44bbc653d8f61e9e2d)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:27.372152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2c04b-8de3-4675-b038-b8b02e487887 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:27.473780 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1327/2644] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:27 2022] PUT /identity/v3/projects/52e5f9c40d204fb5944f7c004a236f5b/users/2867ea18e4524536a0784181454e681a/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:27.484225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11b128-8c99-4cab-b9d5-1b2b71bb11e9 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:27.484516 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11b128-8c99-4cab-b9d5-1b2b71bb11e9 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:27.484797 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11b128-8c99-4cab-b9d5-1b2b71bb11e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:27.511227 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2e11b128-8c99-4cab-b9d5-1b2b71bb11e9 None None] MFA Rules not processed for user `2867ea18e4524536a0784181454e681a`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:27.512412 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2e11b128-8c99-4cab-b9d5-1b2b71bb11e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:27.573784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:27.584870 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1328/2645] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:27.596929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11b128-8c99-4cab-b9d5-1b2b71bb11e9 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:27.598321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2e11b128-8c99-4cab-b9d5-1b2b71bb11e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:27.624961 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2aa2c04b-8de3-4675-b038-b8b02e487887 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a3e633eaf2b4a44bbc653d8f61e9e2d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:27.652765 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2aa2c04b-8de3-4675-b038-b8b02e487887 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a3e633eaf2b4a44bbc653d8f61e9e2d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:27.663699 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11b128-8c99-4cab-b9d5-1b2b71bb11e9 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:27.667898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23b3a2-3a37-4d3f-8a7f-1e4c72221027 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:27.669569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23b3a2-3a37-4d3f-8a7f-1e4c72221027 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:27.669569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23b3a2-3a37-4d3f-8a7f-1e4c72221027 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:27.669569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23b3a2-3a37-4d3f-8a7f-1e4c72221027 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:27.678323 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23b3a2-3a37-4d3f-8a7f-1e4c72221027 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:27.679908 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23b3a2-3a37-4d3f-8a7f-1e4c72221027 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:27.686550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:27.689195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1329/2646] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:25:27.690000 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23b3a2-3a37-4d3f-8a7f-1e4c72221027 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:27.691311 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-0a23b3a2-3a37-4d3f-8a7f-1e4c72221027 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:27.743902 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2aa2c04b-8de3-4675-b038-b8b02e487887 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a3e633eaf2b4a44bbc653d8f61e9e2d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:27.744649 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2aa2c04b-8de3-4675-b038-b8b02e487887 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a3e633eaf2b4a44bbc653d8f61e9e2d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:27.745332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2aa2c04b-8de3-4675-b038-b8b02e487887 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a3e633eaf2b4a44bbc653d8f61e9e2d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:27.758394 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1318/2647] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:27 2022] DELETE /identity/v3/users/6a3e633eaf2b4a44bbc653d8f61e9e2d => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:27.759113 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23b3a2-3a37-4d3f-8a7f-1e4c72221027 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:27.759301 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2c04b-8de3-4675-b038-b8b02e487887 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:27.760622 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2aa2c04b-8de3-4675-b038-b8b02e487887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:27.830030 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2c04b-8de3-4675-b038-b8b02e487887 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:27.831880 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d939b9c-43c9-4eff-8aaf-4b58ae7c1dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:27.832751 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d939b9c-43c9-4eff-8aaf-4b58ae7c1dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:27.833042 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d939b9c-43c9-4eff-8aaf-4b58ae7c1dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:27.833320 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d939b9c-43c9-4eff-8aaf-4b58ae7c1dac admin admin] PATH_INFO: `/v3/projects/223b1389dfb44aa08a57e2a1959ac9dd` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:27.834704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d939b9c-43c9-4eff-8aaf-4b58ae7c1dac admin admin] RBAC: Authorizing `identity:delete_project(project_id=223b1389dfb44aa08a57e2a1959ac9dd)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:27.835086 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c8b83-de72-40e5-b0e6-897662ae1675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:27.836014 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c8b83-de72-40e5-b0e6-897662ae1675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:27.836963 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c8b83-de72-40e5-b0e6-897662ae1675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:27.837533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d939b9c-43c9-4eff-8aaf-4b58ae7c1dac admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:27.837753 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c8b83-de72-40e5-b0e6-897662ae1675 admin admin] PATH_INFO: `/v3/users/2867ea18e4524536a0784181454e681a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:27.839013 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c8b83-de72-40e5-b0e6-897662ae1675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2867ea18e4524536a0784181454e681a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:27.840929 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0c8b83-de72-40e5-b0e6-897662ae1675 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:27.965035 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1d939b9c-43c9-4eff-8aaf-4b58ae7c1dac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '223b1389dfb44aa08a57e2a1959ac9dd', 'request_id': 'req-1d939b9c-43c9-4eff-8aaf-4b58ae7c1dac'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:27.971109 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1d939b9c-43c9-4eff-8aaf-4b58ae7c1dac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '223b1389dfb44aa08a57e2a1959ac9dd', 'request_id': 'req-1d939b9c-43c9-4eff-8aaf-4b58ae7c1dac'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:27.976442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1d939b9c-43c9-4eff-8aaf-4b58ae7c1dac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '223b1389dfb44aa08a57e2a1959ac9dd', 'request_id': 'req-1d939b9c-43c9-4eff-8aaf-4b58ae7c1dac'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:28.058304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1d939b9c-43c9-4eff-8aaf-4b58ae7c1dac admin admin] The token cache is being invalidate because project 223b1389dfb44aa08a57e2a1959ac9dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:28.058631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1d939b9c-43c9-4eff-8aaf-4b58ae7c1dac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:28.059288 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-1d939b9c-43c9-4eff-8aaf-4b58ae7c1dac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:28.060982 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1330/2648] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:27 2022] DELETE /identity/v3/projects/223b1389dfb44aa08a57e2a1959ac9dd => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:28.065315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd09b8d-4fbd-4b67-aedc-f56ab498f1c4 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:28.065315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd09b8d-4fbd-4b67-aedc-f56ab498f1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:28.065315 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd09b8d-4fbd-4b67-aedc-f56ab498f1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:28.083102 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-dfd09b8d-4fbd-4b67-aedc-f56ab498f1c4 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:28.084234 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-dfd09b8d-4fbd-4b67-aedc-f56ab498f1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:28.123950 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2e0c8b83-de72-40e5-b0e6-897662ae1675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2867ea18e4524536a0784181454e681a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:28.136625 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2e0c8b83-de72-40e5-b0e6-897662ae1675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2867ea18e4524536a0784181454e681a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:28.142952 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1331/2649] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:28.147264 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380a4e0-6efc-4af7-a0af-ce1c7125672e None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:28.147547 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380a4e0-6efc-4af7-a0af-ce1c7125672e None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:28.147833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380a4e0-6efc-4af7-a0af-ce1c7125672e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:28.169578 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-3380a4e0-6efc-4af7-a0af-ce1c7125672e None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:28.170933 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3380a4e0-6efc-4af7-a0af-ce1c7125672e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:28.179492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1332/2650] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:28.181073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380a4e0-6efc-4af7-a0af-ce1c7125672e None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:28.182624 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-3380a4e0-6efc-4af7-a0af-ce1c7125672e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:28.240811 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380a4e0-6efc-4af7-a0af-ce1c7125672e None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:28.244878 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf68fe-5f7c-49df-a2df-735994762184 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:28.245560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf68fe-5f7c-49df-a2df-735994762184 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:28.245842 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf68fe-5f7c-49df-a2df-735994762184 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:28.246123 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf68fe-5f7c-49df-a2df-735994762184 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:28.246267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2e0c8b83-de72-40e5-b0e6-897662ae1675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2867ea18e4524536a0784181454e681a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:28.246878 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2e0c8b83-de72-40e5-b0e6-897662ae1675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2867ea18e4524536a0784181454e681a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:28.247485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-2e0c8b83-de72-40e5-b0e6-897662ae1675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2867ea18e4524536a0784181454e681a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:28.247922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-aebf68fe-5f7c-49df-a2df-735994762184 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:28.260963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf68fe-5f7c-49df-a2df-735994762184 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:28.262093 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1319/2651] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:27 2022] DELETE /identity/v3/users/2867ea18e4524536a0784181454e681a => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:28.262425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf68fe-5f7c-49df-a2df-735994762184 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:28.263450 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc16259-1295-452c-b0d4-6714025cfae7 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:28.263737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc16259-1295-452c-b0d4-6714025cfae7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:28.264012 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc16259-1295-452c-b0d4-6714025cfae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:28.272264 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1333/2652] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:28.273564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e933e86-3e47-42bf-abb0-a7cf7506f1f1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:28.273937 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e933e86-3e47-42bf-abb0-a7cf7506f1f1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:28.274244 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e933e86-3e47-42bf-abb0-a7cf7506f1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:28.283742 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-8fc16259-1295-452c-b0d4-6714025cfae7 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:28.285057 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8fc16259-1295-452c-b0d4-6714025cfae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:28.292471 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9e933e86-3e47-42bf-abb0-a7cf7506f1f1 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:28.293670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9e933e86-3e47-42bf-abb0-a7cf7506f1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:28.361406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1320/2653] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:28.362484 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc16259-1295-452c-b0d4-6714025cfae7 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:28.373063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc16259-1295-452c-b0d4-6714025cfae7 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:28.378016 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea35f2b-1b11-4ff7-b38e-f3f2d67543e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:28.378716 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea35f2b-1b11-4ff7-b38e-f3f2d67543e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:28.383114 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea35f2b-1b11-4ff7-b38e-f3f2d67543e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:28.383114 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea35f2b-1b11-4ff7-b38e-f3f2d67543e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:28.383114 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea35f2b-1b11-4ff7-b38e-f3f2d67543e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:25:28.383114 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea35f2b-1b11-4ff7-b38e-f3f2d67543e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:28.384108 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1334/2654] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:28.384625 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea35f2b-1b11-4ff7-b38e-f3f2d67543e9 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:28.392774 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e933e86-3e47-42bf-abb0-a7cf7506f1f1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:28.394320 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9e933e86-3e47-42bf-abb0-a7cf7506f1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:28.396037 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1321/2655] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:25:28.401328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea35f2b-1b11-4ff7-b38e-f3f2d67543e9 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:28.411523 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea35f2b-1b11-4ff7-b38e-f3f2d67543e9 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:28.416879 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fb941-264d-433e-9b96-dc72ad2d880e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:28.417528 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fb941-264d-433e-9b96-dc72ad2d880e admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:28.417814 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fb941-264d-433e-9b96-dc72ad2d880e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:28.418095 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57fb941-264d-433e-9b96-dc72ad2d880e admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:28.418831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fb941-264d-433e-9b96-dc72ad2d880e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:28.420737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57fb941-264d-433e-9b96-dc72ad2d880e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:28.425267 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:28.457044 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e933e86-3e47-42bf-abb0-a7cf7506f1f1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:28.461387 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad9517-73ca-4370-9a0f-5c992ed5073a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:28.463122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad9517-73ca-4370-9a0f-5c992ed5073a service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:28.463412 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad9517-73ca-4370-9a0f-5c992ed5073a service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:28.463693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad9517-73ca-4370-9a0f-5c992ed5073a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:28.475779 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad9517-73ca-4370-9a0f-5c992ed5073a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:28.476455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:28.477593 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad9517-73ca-4370-9a0f-5c992ed5073a service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:28.489843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1335/2656] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:25:28.573394 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1322/2657] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:28 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:28.577831 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad9517-73ca-4370-9a0f-5c992ed5073a service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:28.588234 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad9517-73ca-4370-9a0f-5c992ed5073a service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:28.656662 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f524d-0410-4f0c-8a07-50bdaf4f310a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:28.657500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f524d-0410-4f0c-8a07-50bdaf4f310a admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:28.657790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f524d-0410-4f0c-8a07-50bdaf4f310a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:28.658071 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f524d-0410-4f0c-8a07-50bdaf4f310a admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:28.658813 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f524d-0410-4f0c-8a07-50bdaf4f310a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:28.660798 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f524d-0410-4f0c-8a07-50bdaf4f310a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:28.764901 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1336/2658] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:28 2022] POST /identity/v3/users => generated 292 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:28.766531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fb941-264d-433e-9b96-dc72ad2d880e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:28.778195 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57fb941-264d-433e-9b96-dc72ad2d880e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:28.783322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733c756-463d-42be-ab6e-07bc2b9e6073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:28.783995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733c756-463d-42be-ab6e-07bc2b9e6073 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:28.784283 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733c756-463d-42be-ab6e-07bc2b9e6073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:28.784562 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733c756-463d-42be-ab6e-07bc2b9e6073 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:28.785294 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733c756-463d-42be-ab6e-07bc2b9e6073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:28.787541 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733c756-463d-42be-ab6e-07bc2b9e6073 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:28.802018 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f524d-0410-4f0c-8a07-50bdaf4f310a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:28.806299 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1323/2659] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:28.824244 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733c756-463d-42be-ab6e-07bc2b9e6073 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:28.824244 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f733c756-463d-42be-ab6e-07bc2b9e6073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:28.824816 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f524d-0410-4f0c-8a07-50bdaf4f310a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:28.824816 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f9d2f-b9fe-44f6-8280-66b3ed15bff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:28.824816 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f9d2f-b9fe-44f6-8280-66b3ed15bff5 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:28.824816 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f9d2f-b9fe-44f6-8280-66b3ed15bff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:28.825557 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f9d2f-b9fe-44f6-8280-66b3ed15bff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:28.825557 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f9d2f-b9fe-44f6-8280-66b3ed15bff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:28.825557 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f9d2f-b9fe-44f6-8280-66b3ed15bff5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:28.826498 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733c756-463d-42be-ab6e-07bc2b9e6073 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:28.832228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c880f-4dd9-44c3-a401-6bb6b337e1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:28.833355 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c880f-4dd9-44c3-a401-6bb6b337e1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:28.833747 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c880f-4dd9-44c3-a401-6bb6b337e1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:28.834131 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c880f-4dd9-44c3-a401-6bb6b337e1a2 admin admin] PATH_INFO: `/v3/projects/31e28df63efa487a8bb4d3000f3c6e81` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:28.835548 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c880f-4dd9-44c3-a401-6bb6b337e1a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31e28df63efa487a8bb4d3000f3c6e81)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:28.837905 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1337/2660] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:25:28.838956 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f9d2f-b9fe-44f6-8280-66b3ed15bff5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:28.839495 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c880f-4dd9-44c3-a401-6bb6b337e1a2 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:28.865484 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f9d2f-b9fe-44f6-8280-66b3ed15bff5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:28.879201 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6091e3-9b27-45f5-a889-b3e4420fb116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:28.880261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6091e3-9b27-45f5-a889-b3e4420fb116 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:28.880586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6091e3-9b27-45f5-a889-b3e4420fb116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:28.880938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6091e3-9b27-45f5-a889-b3e4420fb116 admin admin] PATH_INFO: `/v3/projects/2ef415ea2ce84b9591a449dda4b59e7c/users/0d08bbfc0a5a4e32bf7dcfad45c47357/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:28.899763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6091e3-9b27-45f5-a889-b3e4420fb116 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ef415ea2ce84b9591a449dda4b59e7c, user_id=0d08bbfc0a5a4e32bf7dcfad45c47357, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:28.903416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6091e3-9b27-45f5-a889-b3e4420fb116 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:29.017944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1338/2661] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:28 2022] PUT /identity/v3/projects/2ef415ea2ce84b9591a449dda4b59e7c/users/0d08bbfc0a5a4e32bf7dcfad45c47357/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:29.029610 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae32f68-7955-4d3b-9baf-eb23a4ebdad5 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:29.029906 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae32f68-7955-4d3b-9baf-eb23a4ebdad5 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:29.030190 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae32f68-7955-4d3b-9baf-eb23a4ebdad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:29.031493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc7c880f-4dd9-44c3-a401-6bb6b337e1a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31e28df63efa487a8bb4d3000f3c6e81', 'request_id': 'req-dc7c880f-4dd9-44c3-a401-6bb6b337e1a2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:29.061616 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9ae32f68-7955-4d3b-9baf-eb23a4ebdad5 None None] MFA Rules not processed for user `0d08bbfc0a5a4e32bf7dcfad45c47357`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:29.063429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9ae32f68-7955-4d3b-9baf-eb23a4ebdad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:29.104903 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:29.139413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1339/2662] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:29.140192 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae32f68-7955-4d3b-9baf-eb23a4ebdad5 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:29.141455 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9ae32f68-7955-4d3b-9baf-eb23a4ebdad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:29.195356 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc7c880f-4dd9-44c3-a401-6bb6b337e1a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31e28df63efa487a8bb4d3000f3c6e81', 'request_id': 'req-dc7c880f-4dd9-44c3-a401-6bb6b337e1a2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:29.201980 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc7c880f-4dd9-44c3-a401-6bb6b337e1a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31e28df63efa487a8bb4d3000f3c6e81', 'request_id': 'req-dc7c880f-4dd9-44c3-a401-6bb6b337e1a2'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:29.207470 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae32f68-7955-4d3b-9baf-eb23a4ebdad5 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:29.211887 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad09bc7-ebb3-487b-a437-c4296881e5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:29.212883 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad09bc7-ebb3-487b-a437-c4296881e5bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:29.213202 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad09bc7-ebb3-487b-a437-c4296881e5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:29.213486 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad09bc7-ebb3-487b-a437-c4296881e5bb admin admin] PATH_INFO: `/v3/projects/52e5f9c40d204fb5944f7c004a236f5b` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:29.214451 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc7c880f-4dd9-44c3-a401-6bb6b337e1a2 admin admin] The token cache is being invalidate because project 31e28df63efa487a8bb4d3000f3c6e81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:29.214738 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad09bc7-ebb3-487b-a437-c4296881e5bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=52e5f9c40d204fb5944f7c004a236f5b)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:29.214850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc7c880f-4dd9-44c3-a401-6bb6b337e1a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:29.215406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-dc7c880f-4dd9-44c3-a401-6bb6b337e1a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:29.216852 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad09bc7-ebb3-487b-a437-c4296881e5bb admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:29.216990 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1324/2663] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:28 2022] DELETE /identity/v3/projects/31e28df63efa487a8bb4d3000f3c6e81 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:29.217704 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c880f-4dd9-44c3-a401-6bb6b337e1a2 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:29.219000 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dc7c880f-4dd9-44c3-a401-6bb6b337e1a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:29.233611 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c880f-4dd9-44c3-a401-6bb6b337e1a2 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:29.238593 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b9e2f-71ff-4b0d-819c-96cb620f3808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:29.239337 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b9e2f-71ff-4b0d-819c-96cb620f3808 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:29.239641 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b9e2f-71ff-4b0d-819c-96cb620f3808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:29.239902 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b9e2f-71ff-4b0d-819c-96cb620f3808 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:29.240638 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b9e2f-71ff-4b0d-819c-96cb620f3808 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:29.242695 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b9e2f-71ff-4b0d-819c-96cb620f3808 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:29.350659 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1325/2664] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:29 2022] POST /identity/v3/projects => generated 381 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:29.351529 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b9e2f-71ff-4b0d-819c-96cb620f3808 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:29.353916 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a02b9e2f-71ff-4b0d-819c-96cb620f3808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:29.363584 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6ad09bc7-ebb3-487b-a437-c4296881e5bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52e5f9c40d204fb5944f7c004a236f5b', 'request_id': 'req-6ad09bc7-ebb3-487b-a437-c4296881e5bb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:29.371883 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6ad09bc7-ebb3-487b-a437-c4296881e5bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52e5f9c40d204fb5944f7c004a236f5b', 'request_id': 'req-6ad09bc7-ebb3-487b-a437-c4296881e5bb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:29.378725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6ad09bc7-ebb3-487b-a437-c4296881e5bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52e5f9c40d204fb5944f7c004a236f5b', 'request_id': 'req-6ad09bc7-ebb3-487b-a437-c4296881e5bb'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:29.390168 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:29.404884 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:29.426488 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b9e2f-71ff-4b0d-819c-96cb620f3808 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:29.431309 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4fda0-e50f-4b38-8b90-da8d1e785e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:29.434213 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4fda0-e50f-4b38-8b90-da8d1e785e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:29.434213 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4fda0-e50f-4b38-8b90-da8d1e785e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:29.434213 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4fda0-e50f-4b38-8b90-da8d1e785e9e admin admin] PATH_INFO: `/v3/users/be588ea03fff4eccb172e2bbaa20ffb5` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:29.434213 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4fda0-e50f-4b38-8b90-da8d1e785e9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=be588ea03fff4eccb172e2bbaa20ffb5)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:29.436009 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4fda0-e50f-4b38-8b90-da8d1e785e9e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:29.455768 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6ad09bc7-ebb3-487b-a437-c4296881e5bb admin admin] The token cache is being invalidate because project 52e5f9c40d204fb5944f7c004a236f5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:29.456062 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6ad09bc7-ebb3-487b-a437-c4296881e5bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:29.456742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-6ad09bc7-ebb3-487b-a437-c4296881e5bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:29.463134 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1340/2665] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:29 2022] DELETE /identity/v3/projects/52e5f9c40d204fb5944f7c004a236f5b => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:29.463134 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad09bc7-ebb3-487b-a437-c4296881e5bb admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:29.463134 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6ad09bc7-ebb3-487b-a437-c4296881e5bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:29.527940 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad09bc7-ebb3-487b-a437-c4296881e5bb admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:29.532969 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393373e-8f43-41ec-82e3-76a610b7b8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:29.533633 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393373e-8f43-41ec-82e3-76a610b7b8d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:29.533917 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393373e-8f43-41ec-82e3-76a610b7b8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:29.534193 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393373e-8f43-41ec-82e3-76a610b7b8d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:29.534921 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393373e-8f43-41ec-82e3-76a610b7b8d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:29.536841 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3393373e-8f43-41ec-82e3-76a610b7b8d5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:29.703908 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1341/2666] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:29 2022] POST /identity/v3/users => generated 297 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:29.705258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546184b0-73f5-4ba4-9b7e-79b22d757fa7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:29.705556 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546184b0-73f5-4ba4-9b7e-79b22d757fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:29.705838 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546184b0-73f5-4ba4-9b7e-79b22d757fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:29.723794 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-546184b0-73f5-4ba4-9b7e-79b22d757fa7 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:29.726031 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-546184b0-73f5-4ba4-9b7e-79b22d757fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:29.735217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1342/2667] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:29.741847 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546184b0-73f5-4ba4-9b7e-79b22d757fa7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:29.746573 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546184b0-73f5-4ba4-9b7e-79b22d757fa7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:29.751255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c9e4fda0-e50f-4b38-8b90-da8d1e785e9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be588ea03fff4eccb172e2bbaa20ffb5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:29.751525 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3ab23-06e0-4ed6-96bf-d195821e64df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:29.752258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3ab23-06e0-4ed6-96bf-d195821e64df admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:29.752549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3ab23-06e0-4ed6-96bf-d195821e64df admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:29.752823 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3ab23-06e0-4ed6-96bf-d195821e64df admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:29.753547 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3ab23-06e0-4ed6-96bf-d195821e64df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:29.755210 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd3ab23-06e0-4ed6-96bf-d195821e64df admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:29.763890 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c9e4fda0-e50f-4b38-8b90-da8d1e785e9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be588ea03fff4eccb172e2bbaa20ffb5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:29.767210 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1343/2668] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:29.767210 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51d973-367a-48b1-a23e-90e5d67d67e7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:29.772700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51d973-367a-48b1-a23e-90e5d67d67e7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:29.772700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51d973-367a-48b1-a23e-90e5d67d67e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:29.786087 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-4f51d973-367a-48b1-a23e-90e5d67d67e7 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:29.787317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4f51d973-367a-48b1-a23e-90e5d67d67e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:29.796325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1344/2669] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:29.797057 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51d973-367a-48b1-a23e-90e5d67d67e7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:29.807070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51d973-367a-48b1-a23e-90e5d67d67e7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:29.811588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa6049-5213-4d35-b3dd-5c4b4da991fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:29.812446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa6049-5213-4d35-b3dd-5c4b4da991fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:29.812727 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa6049-5213-4d35-b3dd-5c4b4da991fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:29.813003 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa6049-5213-4d35-b3dd-5c4b4da991fd admin admin] PATH_INFO: `/v3/projects/689e29647b3d46c29d0ce35362f05f5e/users/efc77d2274bf456195d03410f4eec88c/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:29.827724 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa6049-5213-4d35-b3dd-5c4b4da991fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=689e29647b3d46c29d0ce35362f05f5e, user_id=efc77d2274bf456195d03410f4eec88c, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:29.830484 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fa6049-5213-4d35-b3dd-5c4b4da991fd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:29.855918 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c9e4fda0-e50f-4b38-8b90-da8d1e785e9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be588ea03fff4eccb172e2bbaa20ffb5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:29.856553 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c9e4fda0-e50f-4b38-8b90-da8d1e785e9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be588ea03fff4eccb172e2bbaa20ffb5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:29.857159 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c9e4fda0-e50f-4b38-8b90-da8d1e785e9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be588ea03fff4eccb172e2bbaa20ffb5'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:29.870900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1326/2670] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:29 2022] DELETE /identity/v3/users/be588ea03fff4eccb172e2bbaa20ffb5 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:29.871770 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4fda0-e50f-4b38-8b90-da8d1e785e9e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:29.874243 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c9e4fda0-e50f-4b38-8b90-da8d1e785e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:29.939970 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4fda0-e50f-4b38-8b90-da8d1e785e9e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:29.944739 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33615234-f393-48c2-bd6e-ed709a89f008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:29.945465 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33615234-f393-48c2-bd6e-ed709a89f008 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:29.946357 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33615234-f393-48c2-bd6e-ed709a89f008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:29.946496 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33615234-f393-48c2-bd6e-ed709a89f008 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:29.947300 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33615234-f393-48c2-bd6e-ed709a89f008 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:25:29.947618 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33615234-f393-48c2-bd6e-ed709a89f008 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:29.949255 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33615234-f393-48c2-bd6e-ed709a89f008 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:29.973787 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1327/2671] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:25:29.974156 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1345/2672] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:29 2022] PUT /identity/v3/projects/689e29647b3d46c29d0ce35362f05f5e/users/efc77d2274bf456195d03410f4eec88c/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:29.974273 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33615234-f393-48c2-bd6e-ed709a89f008 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:29.974392 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa6049-5213-4d35-b3dd-5c4b4da991fd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:29.974392 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-42fa6049-5213-4d35-b3dd-5c4b4da991fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:29.981625 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33615234-f393-48c2-bd6e-ed709a89f008 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:30.037772 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa6049-5213-4d35-b3dd-5c4b4da991fd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:30.038471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484fba6-deab-4ef5-9597-ceaf98307ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:30.039143 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484fba6-deab-4ef5-9597-ceaf98307ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:30.039550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484fba6-deab-4ef5-9597-ceaf98307ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:30.039865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484fba6-deab-4ef5-9597-ceaf98307ab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:30.040598 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484fba6-deab-4ef5-9597-ceaf98307ab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:30.042967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484fba6-deab-4ef5-9597-ceaf98307ab8 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:30.043970 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c984bc-e885-4934-aaa5-3ea84d7a0506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:30.044644 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c984bc-e885-4934-aaa5-3ea84d7a0506 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:30.044930 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c984bc-e885-4934-aaa5-3ea84d7a0506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:30.045211 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c984bc-e885-4934-aaa5-3ea84d7a0506 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:30.045944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c984bc-e885-4934-aaa5-3ea84d7a0506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:30.047579 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c984bc-e885-4934-aaa5-3ea84d7a0506 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:30.060175 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1346/2673] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:30.064027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c984bc-e885-4934-aaa5-3ea84d7a0506 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:30.069595 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:30.075736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c984bc-e885-4934-aaa5-3ea84d7a0506 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:30.079283 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265fc4a-471c-4a7a-9711-e8b5b01a9fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:30.080238 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265fc4a-471c-4a7a-9711-e8b5b01a9fa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:30.080530 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265fc4a-471c-4a7a-9711-e8b5b01a9fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:30.080812 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265fc4a-471c-4a7a-9711-e8b5b01a9fa8 admin admin] PATH_INFO: `/v3/projects/689e29647b3d46c29d0ce35362f05f5e/users/efc77d2274bf456195d03410f4eec88c/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:30.082888 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265fc4a-471c-4a7a-9711-e8b5b01a9fa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=689e29647b3d46c29d0ce35362f05f5e, user_id=efc77d2274bf456195d03410f4eec88c, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:30.085591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265fc4a-471c-4a7a-9711-e8b5b01a9fa8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:30.097022 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1347/2674] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:30 2022] PUT /identity/v3/projects/689e29647b3d46c29d0ce35362f05f5e/users/efc77d2274bf456195d03410f4eec88c/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:30.108921 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3e70e-b3e7-4c3f-9e90-a03da5a12d88 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:30.109518 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3e70e-b3e7-4c3f-9e90-a03da5a12d88 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:30.109606 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3e70e-b3e7-4c3f-9e90-a03da5a12d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:30.136918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-e5a3e70e-b3e7-4c3f-9e90-a03da5a12d88 None None] MFA Rules not processed for user `efc77d2274bf456195d03410f4eec88c`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:30.138133 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e5a3e70e-b3e7-4c3f-9e90-a03da5a12d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:30.154767 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1328/2675] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:29 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:30.158473 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484fba6-deab-4ef5-9597-ceaf98307ab8 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:30.201221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:30.215822 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1348/2676] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:30.229003 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3e70e-b3e7-4c3f-9e90-a03da5a12d88 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:30.230014 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e5a3e70e-b3e7-4c3f-9e90-a03da5a12d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:30.231302 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484fba6-deab-4ef5-9597-ceaf98307ab8 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:30.240249 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f60dc5-fbcf-4d64-87e1-44b384e95b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:30.240945 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f60dc5-fbcf-4d64-87e1-44b384e95b4b admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:30.241351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f60dc5-fbcf-4d64-87e1-44b384e95b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:30.241636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f60dc5-fbcf-4d64-87e1-44b384e95b4b admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:30.242362 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f60dc5-fbcf-4d64-87e1-44b384e95b4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:30.247290 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f60dc5-fbcf-4d64-87e1-44b384e95b4b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:30.291793 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3e70e-b3e7-4c3f-9e90-a03da5a12d88 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:30.343755 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1329/2677] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:30 2022] POST /identity/v3/users => generated 291 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:30.344878 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f60dc5-fbcf-4d64-87e1-44b384e95b4b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:30.346929 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d6f60dc5-fbcf-4d64-87e1-44b384e95b4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:30.347446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca54d0-ac82-4f13-80ca-fd8dcc84e585 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:30.348115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca54d0-ac82-4f13-80ca-fd8dcc84e585 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:30.348403 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca54d0-ac82-4f13-80ca-fd8dcc84e585 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:30.348698 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ca54d0-ac82-4f13-80ca-fd8dcc84e585 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:30.350700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-58ca54d0-ac82-4f13-80ca-fd8dcc84e585 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:30.363422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f60dc5-fbcf-4d64-87e1-44b384e95b4b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:30.371002 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255a454-bd19-476e-a591-4b5357728591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:30.371881 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255a454-bd19-476e-a591-4b5357728591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:30.372266 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255a454-bd19-476e-a591-4b5357728591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:30.372547 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255a454-bd19-476e-a591-4b5357728591 admin admin] PATH_INFO: `/v3/projects/197df0cb183f40c681f824473b8112ee` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:30.373769 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255a454-bd19-476e-a591-4b5357728591 admin admin] RBAC: Authorizing `identity:delete_project(project_id=197df0cb183f40c681f824473b8112ee)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:30.376181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255a454-bd19-476e-a591-4b5357728591 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:30.418483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca54d0-ac82-4f13-80ca-fd8dcc84e585 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:30.420091 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ca54d0-ac82-4f13-80ca-fd8dcc84e585 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:30.429522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1349/2678] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:30.431089 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca54d0-ac82-4f13-80ca-fd8dcc84e585 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:30.442070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ca54d0-ac82-4f13-80ca-fd8dcc84e585 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:30.447785 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ad069-c57f-4325-aa0c-d83a4f4ddd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:30.448453 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ad069-c57f-4325-aa0c-d83a4f4ddd91 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:30.448736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ad069-c57f-4325-aa0c-d83a4f4ddd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:30.449010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ad069-c57f-4325-aa0c-d83a4f4ddd91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:30.449745 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ad069-c57f-4325-aa0c-d83a4f4ddd91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:30.451403 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ad069-c57f-4325-aa0c-d83a4f4ddd91 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:30.462418 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1350/2679] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:30.466240 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ad069-c57f-4325-aa0c-d83a4f4ddd91 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:30.477908 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ad069-c57f-4325-aa0c-d83a4f4ddd91 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:30.494241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f330f1-a347-457f-a1ec-ba042c6c29d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:30.494241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f330f1-a347-457f-a1ec-ba042c6c29d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:30.494241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f330f1-a347-457f-a1ec-ba042c6c29d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:30.494241 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f330f1-a347-457f-a1ec-ba042c6c29d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:30.494907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f330f1-a347-457f-a1ec-ba042c6c29d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:30.496795 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f330f1-a347-457f-a1ec-ba042c6c29d5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:30.510450 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1351/2680] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:25:30.513063 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f330f1-a347-457f-a1ec-ba042c6c29d5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:30.518627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a255a454-bd19-476e-a591-4b5357728591 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '197df0cb183f40c681f824473b8112ee', 'request_id': 'req-a255a454-bd19-476e-a591-4b5357728591'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:30.527674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f330f1-a347-457f-a1ec-ba042c6c29d5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:30.589296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a255a454-bd19-476e-a591-4b5357728591 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '197df0cb183f40c681f824473b8112ee', 'request_id': 'req-a255a454-bd19-476e-a591-4b5357728591'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:30.595928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70682581-c425-4550-82fb-8b65fa3d438f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:30.596085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a255a454-bd19-476e-a591-4b5357728591 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '197df0cb183f40c681f824473b8112ee', 'request_id': 'req-a255a454-bd19-476e-a591-4b5357728591'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:30.599177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70682581-c425-4550-82fb-8b65fa3d438f admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:30.599177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70682581-c425-4550-82fb-8b65fa3d438f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:30.599177 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70682581-c425-4550-82fb-8b65fa3d438f admin admin] PATH_INFO: `/v3/projects/2a191785599744ef907f9e1f17bc572d/users/7b0c3b26ed4e4f128e780460c0bfd434/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:30.609197 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a255a454-bd19-476e-a591-4b5357728591 admin admin] The token cache is being invalidate because project 197df0cb183f40c681f824473b8112ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:30.609580 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a255a454-bd19-476e-a591-4b5357728591 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:30.610448 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a255a454-bd19-476e-a591-4b5357728591 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:30.612101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1330/2681] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:30 2022] DELETE /identity/v3/projects/197df0cb183f40c681f824473b8112ee => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:30.616841 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0e985-3d70-4e0c-a933-5f36a39cf7ae None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:30.617134 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0e985-3d70-4e0c-a933-5f36a39cf7ae None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:30.617420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0e985-3d70-4e0c-a933-5f36a39cf7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:30.620265 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70682581-c425-4550-82fb-8b65fa3d438f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a191785599744ef907f9e1f17bc572d, user_id=7b0c3b26ed4e4f128e780460c0bfd434, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:30.623062 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70682581-c425-4550-82fb-8b65fa3d438f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:30.637674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-ffb0e985-3d70-4e0c-a933-5f36a39cf7ae None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:30.639059 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-ffb0e985-3d70-4e0c-a933-5f36a39cf7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:30.646556 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:30.647899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1331/2682] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:30.657977 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e94712-ac2a-4953-ae3b-f6e2671549be None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:30.658426 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e94712-ac2a-4953-ae3b-f6e2671549be None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:30.658762 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e94712-ac2a-4953-ae3b-f6e2671549be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:30.673647 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1352/2683] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:30 2022] PUT /identity/v3/projects/2a191785599744ef907f9e1f17bc572d/users/7b0c3b26ed4e4f128e780460c0bfd434/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:30.677407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-60e94712-ac2a-4953-ae3b-f6e2671549be None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:30.678645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-60e94712-ac2a-4953-ae3b-f6e2671549be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:30.710470 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:30.739169 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ab92d-0051-47eb-8041-b23d38d5539d None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:30.739463 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ab92d-0051-47eb-8041-b23d38d5539d None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:30.739745 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ab92d-0051-47eb-8041-b23d38d5539d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:30.747708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1332/2684] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:30.751676 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e94712-ac2a-4953-ae3b-f6e2671549be None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:30.761495 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e94712-ac2a-4953-ae3b-f6e2671549be None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:30.766171 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86908d1-8a25-447a-bf96-3bb3d037cae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:30.766857 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86908d1-8a25-447a-bf96-3bb3d037cae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:30.767140 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86908d1-8a25-447a-bf96-3bb3d037cae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:30.767420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86908d1-8a25-447a-bf96-3bb3d037cae6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:30.768169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86908d1-8a25-447a-bf96-3bb3d037cae6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:25:30.768484 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86908d1-8a25-447a-bf96-3bb3d037cae6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:30.769034 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-941ab92d-0051-47eb-8041-b23d38d5539d None None] MFA Rules not processed for user `7b0c3b26ed4e4f128e780460c0bfd434`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:30.770085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86908d1-8a25-447a-bf96-3bb3d037cae6 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:30.770281 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-941ab92d-0051-47eb-8041-b23d38d5539d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:30.782737 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1333/2685] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:25:30.787004 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86908d1-8a25-447a-bf96-3bb3d037cae6 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:30.796805 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86908d1-8a25-447a-bf96-3bb3d037cae6 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:30.801742 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273d5bc-8a42-4642-af66-b4b5cc2bfafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:30.806146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273d5bc-8a42-4642-af66-b4b5cc2bfafa admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:30.806146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273d5bc-8a42-4642-af66-b4b5cc2bfafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:30.806146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273d5bc-8a42-4642-af66-b4b5cc2bfafa admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:30.806146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273d5bc-8a42-4642-af66-b4b5cc2bfafa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:30.806563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273d5bc-8a42-4642-af66-b4b5cc2bfafa admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:30.840051 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1353/2686] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:30.844518 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ab92d-0051-47eb-8041-b23d38d5539d None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:30.846104 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-941ab92d-0051-47eb-8041-b23d38d5539d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:30.859805 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ab92d-0051-47eb-8041-b23d38d5539d None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:30.864540 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c390c-c8b3-4925-a95e-96767457774a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:30.865188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c390c-c8b3-4925-a95e-96767457774a admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:30.865476 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c390c-c8b3-4925-a95e-96767457774a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:30.865756 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596c390c-c8b3-4925-a95e-96767457774a admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:30.866479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c390c-c8b3-4925-a95e-96767457774a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:30.868472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596c390c-c8b3-4925-a95e-96767457774a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:30.887790 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1334/2687] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:30 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:30.891827 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273d5bc-8a42-4642-af66-b4b5cc2bfafa admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:30.953027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273d5bc-8a42-4642-af66-b4b5cc2bfafa admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:30.957799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8f3c7-ec18-4e34-bd99-d4021154a0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:30.958508 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8f3c7-ec18-4e34-bd99-d4021154a0fc admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:30.958845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8f3c7-ec18-4e34-bd99-d4021154a0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:30.959125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8f3c7-ec18-4e34-bd99-d4021154a0fc admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:30.959817 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8f3c7-ec18-4e34-bd99-d4021154a0fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:30.962349 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8f3c7-ec18-4e34-bd99-d4021154a0fc admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:30.978800 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1354/2688] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:30 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:30.982721 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c390c-c8b3-4925-a95e-96767457774a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:31.045705 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596c390c-c8b3-4925-a95e-96767457774a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:31.050606 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96925cb-78a8-493a-a0b8-354bd9486ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:31.051278 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96925cb-78a8-493a-a0b8-354bd9486ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:31.051550 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96925cb-78a8-493a-a0b8-354bd9486ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:31.051830 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96925cb-78a8-493a-a0b8-354bd9486ae2 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:31.052522 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96925cb-78a8-493a-a0b8-354bd9486ae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:31.054483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96925cb-78a8-493a-a0b8-354bd9486ae2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:31.080302 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1335/2689] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:30 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:31.080302 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8f3c7-ec18-4e34-bd99-d4021154a0fc admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:31.088456 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8f3c7-ec18-4e34-bd99-d4021154a0fc admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:31.093115 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df1f14-5f5f-4db3-be3e-5b4313419a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:31.093756 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df1f14-5f5f-4db3-be3e-5b4313419a95 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:31.094046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df1f14-5f5f-4db3-be3e-5b4313419a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:31.094364 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df1f14-5f5f-4db3-be3e-5b4313419a95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:31.095113 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df1f14-5f5f-4db3-be3e-5b4313419a95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:31.096700 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df1f14-5f5f-4db3-be3e-5b4313419a95 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:31.107033 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1336/2690] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:25:31.110917 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df1f14-5f5f-4db3-be3e-5b4313419a95 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:31.120374 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df1f14-5f5f-4db3-be3e-5b4313419a95 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:31.125003 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf0c6d-bdaf-44ad-b103-a638194e1e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:31.125647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf0c6d-bdaf-44ad-b103-a638194e1e22 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:31.125935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf0c6d-bdaf-44ad-b103-a638194e1e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:31.126216 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf0c6d-bdaf-44ad-b103-a638194e1e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:31.126991 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf0c6d-bdaf-44ad-b103-a638194e1e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:31.128594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf0c6d-bdaf-44ad-b103-a638194e1e22 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:31.140669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1337/2691] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:25:31.144487 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf0c6d-bdaf-44ad-b103-a638194e1e22 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:31.154935 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf0c6d-bdaf-44ad-b103-a638194e1e22 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:31.156651 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1355/2692] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:30 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:31.159936 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96925cb-78a8-493a-a0b8-354bd9486ae2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:31.161165 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4cbc71-ddc7-49a6-986f-b89640360380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:31.162044 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cbc71-ddc7-49a6-986f-b89640360380 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:31.162327 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cbc71-ddc7-49a6-986f-b89640360380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:31.162634 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cbc71-ddc7-49a6-986f-b89640360380 admin admin] PATH_INFO: `/v3/projects/ec7dff5f75b64f6cb7109641144c552d/users/f3784b41091646e9b77d4420f6029694/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:31.163852 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:31.176266 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96925cb-78a8-493a-a0b8-354bd9486ae2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:31.186606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4cbc71-ddc7-49a6-986f-b89640360380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec7dff5f75b64f6cb7109641144c552d, user_id=f3784b41091646e9b77d4420f6029694, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:31.186606 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4cbc71-ddc7-49a6-986f-b89640360380 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:31.189304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4af420-b50f-4cb2-9ffa-d80ed046c492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:31.190107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4af420-b50f-4cb2-9ffa-d80ed046c492 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:31.190708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4af420-b50f-4cb2-9ffa-d80ed046c492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:31.190982 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4af420-b50f-4cb2-9ffa-d80ed046c492 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:31.191722 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4af420-b50f-4cb2-9ffa-d80ed046c492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:31.193416 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4af420-b50f-4cb2-9ffa-d80ed046c492 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:31.204972 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1356/2693] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:25:31.208920 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4af420-b50f-4cb2-9ffa-d80ed046c492 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:31.219361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4af420-b50f-4cb2-9ffa-d80ed046c492 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:31.224270 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325de315-8e3f-4528-bcaf-e57160be279a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:31.224932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325de315-8e3f-4528-bcaf-e57160be279a admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:31.225225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325de315-8e3f-4528-bcaf-e57160be279a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:31.225935 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325de315-8e3f-4528-bcaf-e57160be279a admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:31.226588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325de315-8e3f-4528-bcaf-e57160be279a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:31.228264 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325de315-8e3f-4528-bcaf-e57160be279a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:31.239115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1357/2694] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:25:31.243118 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325de315-8e3f-4528-bcaf-e57160be279a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:31.252604 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325de315-8e3f-4528-bcaf-e57160be279a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:31.257214 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61d0a0-2c1b-4563-a6d9-5efa20917704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:31.258070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61d0a0-2c1b-4563-a6d9-5efa20917704 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:31.258950 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61d0a0-2c1b-4563-a6d9-5efa20917704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:31.259230 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61d0a0-2c1b-4563-a6d9-5efa20917704 admin admin] PATH_INFO: `/v3/projects/59ddee588ff346dda41ede6092d6b9c4/users/24bca992801d40da8286bd08a439a2df/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:31.265053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1338/2695] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:31 2022] PUT /identity/v3/projects/ec7dff5f75b64f6cb7109641144c552d/users/f3784b41091646e9b77d4420f6029694/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:31.275852 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61d0a0-2c1b-4563-a6d9-5efa20917704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59ddee588ff346dda41ede6092d6b9c4, user_id=24bca992801d40da8286bd08a439a2df, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:31.278468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368d70f-329b-45fe-be74-5449f5f0be69 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:31.278468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368d70f-329b-45fe-be74-5449f5f0be69 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:31.278468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368d70f-329b-45fe-be74-5449f5f0be69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:31.279465 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61d0a0-2c1b-4563-a6d9-5efa20917704 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:31.305536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d368d70f-329b-45fe-be74-5449f5f0be69 None None] MFA Rules not processed for user `f3784b41091646e9b77d4420f6029694`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:31.306699 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d368d70f-329b-45fe-be74-5449f5f0be69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:31.321158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:31.336422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:31.355063 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1358/2696] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:31 2022] PUT /identity/v3/projects/59ddee588ff346dda41ede6092d6b9c4/users/24bca992801d40da8286bd08a439a2df/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:31.365362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7745f3-7874-4658-86ef-4f0f3b398cc1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:31.365679 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7745f3-7874-4658-86ef-4f0f3b398cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:31.365963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7745f3-7874-4658-86ef-4f0f3b398cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:31.372631 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1339/2697] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:31.379818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368d70f-329b-45fe-be74-5449f5f0be69 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:31.393454 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-de7745f3-7874-4658-86ef-4f0f3b398cc1 None None] MFA Rules not processed for user `24bca992801d40da8286bd08a439a2df`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:31.394764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-de7745f3-7874-4658-86ef-4f0f3b398cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:31.444134 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d368d70f-329b-45fe-be74-5449f5f0be69 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:31.462023 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1359/2698] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:31.466138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7745f3-7874-4658-86ef-4f0f3b398cc1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:31.480996 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:31.503663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb726c2e-0e9a-45be-a0b3-afc0880a3ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:31.504324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb726c2e-0e9a-45be-a0b3-afc0880a3ce1 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:31.504633 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb726c2e-0e9a-45be-a0b3-afc0880a3ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:31.504908 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb726c2e-0e9a-45be-a0b3-afc0880a3ce1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:31.505594 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb726c2e-0e9a-45be-a0b3-afc0880a3ce1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:31.508498 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb726c2e-0e9a-45be-a0b3-afc0880a3ce1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:31.530957 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7745f3-7874-4658-86ef-4f0f3b398cc1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:31.535652 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f77692-6838-4f25-928b-890b912a62de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:31.537322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f77692-6838-4f25-928b-890b912a62de admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:31.537322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f77692-6838-4f25-928b-890b912a62de admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:31.537322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f77692-6838-4f25-928b-890b912a62de admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:31.537622 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f77692-6838-4f25-928b-890b912a62de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:31.540086 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f77692-6838-4f25-928b-890b912a62de admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:31.591707 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1340/2699] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:31 2022] POST /identity/v3/projects => generated 381 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:31.595217 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb726c2e-0e9a-45be-a0b3-afc0880a3ce1 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:31.647490 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1360/2700] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:31 2022] POST /identity/v3/projects => generated 383 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:31.652729 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f77692-6838-4f25-928b-890b912a62de admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:31.661445 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb726c2e-0e9a-45be-a0b3-afc0880a3ce1 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:31.666218 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:31.716640 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f77692-6838-4f25-928b-890b912a62de admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:31.719688 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354c82c-d820-474e-a9c0-c3b50788842d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:31.720370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354c82c-d820-474e-a9c0-c3b50788842d admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:31.720664 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354c82c-d820-474e-a9c0-c3b50788842d admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:31.720946 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354c82c-d820-474e-a9c0-c3b50788842d admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:31.721659 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354c82c-d820-474e-a9c0-c3b50788842d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:31.722866 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563441db-0b95-4f21-bcde-cf546736884a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:31.723630 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354c82c-d820-474e-a9c0-c3b50788842d admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:31.724012 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563441db-0b95-4f21-bcde-cf546736884a admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:31.724304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563441db-0b95-4f21-bcde-cf546736884a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:31.724584 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563441db-0b95-4f21-bcde-cf546736884a admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:31.725346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563441db-0b95-4f21-bcde-cf546736884a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:31.727325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563441db-0b95-4f21-bcde-cf546736884a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:31.811622 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1361/2701] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:31 2022] POST /identity/v3/users => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:31.815499 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563441db-0b95-4f21-bcde-cf546736884a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:31.815885 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1341/2702] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:31 2022] POST /identity/v3/users => generated 297 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:31.816866 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354c82c-d820-474e-a9c0-c3b50788842d admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:31.825965 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563441db-0b95-4f21-bcde-cf546736884a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:31.827579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354c82c-d820-474e-a9c0-c3b50788842d admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:31.830701 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210cdcca-f702-42f6-b2d3-b403bcad0fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:31.834513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cdcca-f702-42f6-b2d3-b403bcad0fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:31.834513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cdcca-f702-42f6-b2d3-b403bcad0fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:31.834513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cdcca-f702-42f6-b2d3-b403bcad0fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:31.834513 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210cdcca-f702-42f6-b2d3-b403bcad0fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:31.834968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f503ed7-45bd-4cef-adfe-7e9584b97a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:31.834968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f503ed7-45bd-4cef-adfe-7e9584b97a47 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:31.834968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f503ed7-45bd-4cef-adfe-7e9584b97a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:31.834968 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f503ed7-45bd-4cef-adfe-7e9584b97a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:31.835350 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210cdcca-f702-42f6-b2d3-b403bcad0fa3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:31.835456 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f503ed7-45bd-4cef-adfe-7e9584b97a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:31.836813 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f503ed7-45bd-4cef-adfe-7e9584b97a47 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:31.844461 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1362/2703] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:25:31.850201 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1342/2704] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:31.850361 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210cdcca-f702-42f6-b2d3-b403bcad0fa3 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:31.851179 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f503ed7-45bd-4cef-adfe-7e9584b97a47 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:31.858334 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210cdcca-f702-42f6-b2d3-b403bcad0fa3 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:31.863322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f503ed7-45bd-4cef-adfe-7e9584b97a47 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:31.863442 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e74a1-5460-4d82-8422-6f86a6fd4a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:31.863900 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e74a1-5460-4d82-8422-6f86a6fd4a0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:31.864184 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e74a1-5460-4d82-8422-6f86a6fd4a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:31.864464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1e74a1-5460-4d82-8422-6f86a6fd4a0d admin admin] PATH_INFO: `/v3/projects/9e69408a96c948b18c3456d4254a9930/users/d9516f538b7d46549e3f0b9c0159001f/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:31.867406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa3e99-b600-47ee-b1fd-71e4a838f517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:31.869298 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa3e99-b600-47ee-b1fd-71e4a838f517 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:31.869582 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa3e99-b600-47ee-b1fd-71e4a838f517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:31.869859 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa3e99-b600-47ee-b1fd-71e4a838f517 admin admin] PATH_INFO: `/v3/projects/e62cf048a94841c3b0642d5da05fcc63/users/c941478608544f7799bd3cf9a7b733cd/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:31.880527 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e74a1-5460-4d82-8422-6f86a6fd4a0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e69408a96c948b18c3456d4254a9930, user_id=d9516f538b7d46549e3f0b9c0159001f, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:31.884472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1e74a1-5460-4d82-8422-6f86a6fd4a0d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:31.885503 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa3e99-b600-47ee-b1fd-71e4a838f517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e62cf048a94841c3b0642d5da05fcc63, user_id=c941478608544f7799bd3cf9a7b733cd, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:31.888192 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa3e99-b600-47ee-b1fd-71e4a838f517 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:31.968466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:31.973191 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1363/2705] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:31 2022] PUT /identity/v3/projects/9e69408a96c948b18c3456d4254a9930/users/d9516f538b7d46549e3f0b9c0159001f/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:31.978256 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e74a1-5460-4d82-8422-6f86a6fd4a0d admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:31.984491 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1343/2706] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:31 2022] PUT /identity/v3/projects/e62cf048a94841c3b0642d5da05fcc63/users/c941478608544f7799bd3cf9a7b733cd/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:31.988259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa3e99-b600-47ee-b1fd-71e4a838f517 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:32.041554 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1e74a1-5460-4d82-8422-6f86a6fd4a0d admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:32.063324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa3e99-b600-47ee-b1fd-71e4a838f517 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:32.069550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f09f3-a7dc-47a0-b4fb-8c805e2f3e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:32.069550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f09f3-a7dc-47a0-b4fb-8c805e2f3e80 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:32.069550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f09f3-a7dc-47a0-b4fb-8c805e2f3e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:32.069550 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f09f3-a7dc-47a0-b4fb-8c805e2f3e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:32.069963 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f09f3-a7dc-47a0-b4fb-8c805e2f3e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:32.071503 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f09f3-a7dc-47a0-b4fb-8c805e2f3e80 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:32.081593 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1344/2707] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:25:32.085574 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f09f3-a7dc-47a0-b4fb-8c805e2f3e80 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:32.095085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f09f3-a7dc-47a0-b4fb-8c805e2f3e80 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:32.099597 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bdb73-83cf-4973-9f2e-1b13abb5bc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:32.100274 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfe340-e5a6-4314-bd48-ff7107558d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:32.100448 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bdb73-83cf-4973-9f2e-1b13abb5bc71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:32.100791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bdb73-83cf-4973-9f2e-1b13abb5bc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:32.100946 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfe340-e5a6-4314-bd48-ff7107558d77 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:32.101068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bdb73-83cf-4973-9f2e-1b13abb5bc71 admin admin] PATH_INFO: `/v3/projects/e62cf048a94841c3b0642d5da05fcc63/users/c941478608544f7799bd3cf9a7b733cd/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:32.101235 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfe340-e5a6-4314-bd48-ff7107558d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:32.101517 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cfe340-e5a6-4314-bd48-ff7107558d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:32.103071 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cfe340-e5a6-4314-bd48-ff7107558d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:32.103563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bdb73-83cf-4973-9f2e-1b13abb5bc71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e62cf048a94841c3b0642d5da05fcc63, user_id=c941478608544f7799bd3cf9a7b733cd, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:32.104678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cfe340-e5a6-4314-bd48-ff7107558d77 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:32.106175 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191bdb73-83cf-4973-9f2e-1b13abb5bc71 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:32.114487 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1364/2708] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:25:32.118673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfe340-e5a6-4314-bd48-ff7107558d77 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:32.120191 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1345/2709] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:32 2022] PUT /identity/v3/projects/e62cf048a94841c3b0642d5da05fcc63/users/c941478608544f7799bd3cf9a7b733cd/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:32.128228 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cfe340-e5a6-4314-bd48-ff7107558d77 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:32.130008 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfd2cb-fcd8-4d9c-973a-bc85150b8670 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:32.130308 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfd2cb-fcd8-4d9c-973a-bc85150b8670 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:32.130632 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cfd2cb-fcd8-4d9c-973a-bc85150b8670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:32.159122 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-38cfd2cb-fcd8-4d9c-973a-bc85150b8670 None None] MFA Rules not processed for user `c941478608544f7799bd3cf9a7b733cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:32.160357 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-38cfd2cb-fcd8-4d9c-973a-bc85150b8670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:32.185790 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1548587-c64b-487f-a50b-69359b48d6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:32.186719 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1548587-c64b-487f-a50b-69359b48d6fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:32.187003 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1548587-c64b-487f-a50b-69359b48d6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:32.187277 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1548587-c64b-487f-a50b-69359b48d6fc admin admin] PATH_INFO: `/v3/projects/9e69408a96c948b18c3456d4254a9930/users/d9516f538b7d46549e3f0b9c0159001f/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:32.189278 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1548587-c64b-487f-a50b-69359b48d6fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e69408a96c948b18c3456d4254a9930, user_id=d9516f538b7d46549e3f0b9c0159001f, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:32.191975 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1548587-c64b-487f-a50b-69359b48d6fc admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:32.193833 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:32.203239 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1365/2710] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:32 2022] PUT /identity/v3/projects/9e69408a96c948b18c3456d4254a9930/users/d9516f538b7d46549e3f0b9c0159001f/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:32.222898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cb409-e2a5-43fb-a79b-2ec48a4c83ce None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:32.223185 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cb409-e2a5-43fb-a79b-2ec48a4c83ce None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:32.223460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cb409-e2a5-43fb-a79b-2ec48a4c83ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:32.229280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1346/2711] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:32.243637 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfd2cb-fcd8-4d9c-973a-bc85150b8670 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:32.246575 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-38cfd2cb-fcd8-4d9c-973a-bc85150b8670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:32.257938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-538cb409-e2a5-43fb-a79b-2ec48a4c83ce None None] MFA Rules not processed for user `d9516f538b7d46549e3f0b9c0159001f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:32.257938 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-538cb409-e2a5-43fb-a79b-2ec48a4c83ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:32.292588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:32.316014 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cfd2cb-fcd8-4d9c-973a-bc85150b8670 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:32.344074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1366/2712] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:32 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:32.368116 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e0911-cd55-4c9e-b78c-4e839dfbec0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:32.368891 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e0911-cd55-4c9e-b78c-4e839dfbec0d service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:32.369178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e0911-cd55-4c9e-b78c-4e839dfbec0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:32.369452 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e0911-cd55-4c9e-b78c-4e839dfbec0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:32.429728 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e0911-cd55-4c9e-b78c-4e839dfbec0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:32.431751 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e0911-cd55-4c9e-b78c-4e839dfbec0d service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:32.441407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1347/2713] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:32.481028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538cb409-e2a5-43fb-a79b-2ec48a4c83ce None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:32.490775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538cb409-e2a5-43fb-a79b-2ec48a4c83ce None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:32.495373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b5a09-c4c8-4b32-886f-90a3d75a64ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:32.497152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b5a09-c4c8-4b32-886f-90a3d75a64ac service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:32.497152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b5a09-c4c8-4b32-886f-90a3d75a64ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:32.497152 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b5a09-c4c8-4b32-886f-90a3d75a64ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:32.558272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b5a09-c4c8-4b32-886f-90a3d75a64ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:32.559762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b5a09-c4c8-4b32-886f-90a3d75a64ac service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:32.568631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1367/2714] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:33.787765 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e0911-cd55-4c9e-b78c-4e839dfbec0d service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:33.798094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e0911-cd55-4c9e-b78c-4e839dfbec0d service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:33.802692 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ddf97-57d6-4a3b-9828-cf736ff938a8 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c941478608544f7799bd3cf9a7b733cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e62cf048a94841c3b0642d5da05fcc63', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:33.803439 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ddf97-57d6-4a3b-9828-cf736ff938a8 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:33.803766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ddf97-57d6-4a3b-9828-cf736ff938a8 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:33.804039 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ddf97-57d6-4a3b-9828-cf736ff938a8 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:33.804766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ddf97-57d6-4a3b-9828-cf736ff938a8 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:25:33.805088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ddf97-57d6-4a3b-9828-cf736ff938a8 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:33.806740 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ddf97-57d6-4a3b-9828-cf736ff938a8 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:33.818261 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1348/2715] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:25:33.822888 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b5a09-c4c8-4b32-886f-90a3d75a64ac service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:33.833753 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b5a09-c4c8-4b32-886f-90a3d75a64ac service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:33.839305 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca62b02-aff2-49f2-8bb2-b00011353cc8 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c941478608544f7799bd3cf9a7b733cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e62cf048a94841c3b0642d5da05fcc63', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:33.839988 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca62b02-aff2-49f2-8bb2-b00011353cc8 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:33.840274 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca62b02-aff2-49f2-8bb2-b00011353cc8 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:33.840554 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca62b02-aff2-49f2-8bb2-b00011353cc8 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:33.841255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca62b02-aff2-49f2-8bb2-b00011353cc8 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:33.843248 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca62b02-aff2-49f2-8bb2-b00011353cc8 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:33.970223 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1368/2716] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:33 2022] POST /identity/v3/projects => generated 355 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:33.971871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ddf97-57d6-4a3b-9828-cf736ff938a8 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:33.983065 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ddf97-57d6-4a3b-9828-cf736ff938a8 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:34.006348 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:34.050949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedad2f7-6303-4d48-9339-b353d35b0eb3 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c941478608544f7799bd3cf9a7b733cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e62cf048a94841c3b0642d5da05fcc63', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:34.051617 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedad2f7-6303-4d48-9339-b353d35b0eb3 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:34.051909 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedad2f7-6303-4d48-9339-b353d35b0eb3 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:34.052188 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedad2f7-6303-4d48-9339-b353d35b0eb3 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:34.052917 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedad2f7-6303-4d48-9339-b353d35b0eb3 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:25:34.053231 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedad2f7-6303-4d48-9339-b353d35b0eb3 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:34.054845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedad2f7-6303-4d48-9339-b353d35b0eb3 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:34.067679 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1349/2717] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:25:34.107188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca62b02-aff2-49f2-8bb2-b00011353cc8 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:34.118114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca62b02-aff2-49f2-8bb2-b00011353cc8 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:34.122835 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e44d51-ba69-4c17-b0fb-db09df2b2b0e tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c941478608544f7799bd3cf9a7b733cd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e62cf048a94841c3b0642d5da05fcc63', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:34.123721 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e44d51-ba69-4c17-b0fb-db09df2b2b0e tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:34.124025 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e44d51-ba69-4c17-b0fb-db09df2b2b0e tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:34.124305 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e44d51-ba69-4c17-b0fb-db09df2b2b0e tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] PATH_INFO: `/v3/projects/b85f010114e74a75bf56363074b066e6` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:34.125479 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e44d51-ba69-4c17-b0fb-db09df2b2b0e tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b85f010114e74a75bf56363074b066e6)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:34.127428 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e44d51-ba69-4c17-b0fb-db09df2b2b0e tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:34.217048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-97e44d51-ba69-4c17-b0fb-db09df2b2b0e tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b85f010114e74a75bf56363074b066e6', 'request_id': 'req-97e44d51-ba69-4c17-b0fb-db09df2b2b0e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:34.223726 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-97e44d51-ba69-4c17-b0fb-db09df2b2b0e tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b85f010114e74a75bf56363074b066e6', 'request_id': 'req-97e44d51-ba69-4c17-b0fb-db09df2b2b0e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:34.229912 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-97e44d51-ba69-4c17-b0fb-db09df2b2b0e tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b85f010114e74a75bf56363074b066e6', 'request_id': 'req-97e44d51-ba69-4c17-b0fb-db09df2b2b0e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:34.343961 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-97e44d51-ba69-4c17-b0fb-db09df2b2b0e tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] The token cache is being invalidate because project b85f010114e74a75bf56363074b066e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:34.344273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-97e44d51-ba69-4c17-b0fb-db09df2b2b0e tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:34.344899 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-97e44d51-ba69-4c17-b0fb-db09df2b2b0e tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:34.346577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1369/2718] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:34 2022] DELETE /identity/v3/projects/b85f010114e74a75bf56363074b066e6 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:34.609427 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedad2f7-6303-4d48-9339-b353d35b0eb3 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:34.611590 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dedad2f7-6303-4d48-9339-b353d35b0eb3 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:34.672000 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedad2f7-6303-4d48-9339-b353d35b0eb3 tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:34.721962 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b653f6-bbb0-45e2-a73b-7c2e66aed9be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:34.722637 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b653f6-bbb0-45e2-a73b-7c2e66aed9be service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:34.722921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b653f6-bbb0-45e2-a73b-7c2e66aed9be service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:34.723197 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b653f6-bbb0-45e2-a73b-7c2e66aed9be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:34.725111 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-80b653f6-bbb0-45e2-a73b-7c2e66aed9be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:34.786272 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b653f6-bbb0-45e2-a73b-7c2e66aed9be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:34.787740 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b653f6-bbb0-45e2-a73b-7c2e66aed9be service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:34.796796 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1350/2719] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:35.157493 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e44d51-ba69-4c17-b0fb-db09df2b2b0e tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:35.159053 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-97e44d51-ba69-4c17-b0fb-db09df2b2b0e tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:35.187481 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:35.228080 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e44d51-ba69-4c17-b0fb-db09df2b2b0e tempest-VolumeQuotaClassesTest-1992977632 tempest-VolumeQuotaClassesTest-1992977632-project-admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:35.232884 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30949d3-021b-433a-8c79-2d19cbeed6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:35.233758 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30949d3-021b-433a-8c79-2d19cbeed6bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:35.234053 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30949d3-021b-433a-8c79-2d19cbeed6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:35.234330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30949d3-021b-433a-8c79-2d19cbeed6bd admin admin] PATH_INFO: `/v3/users/f3784b41091646e9b77d4420f6029694` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:35.235591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30949d3-021b-433a-8c79-2d19cbeed6bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3784b41091646e9b77d4420f6029694)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:35.237664 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30949d3-021b-433a-8c79-2d19cbeed6bd admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:35.407217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e30949d3-021b-433a-8c79-2d19cbeed6bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3784b41091646e9b77d4420f6029694'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:35.421512 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e30949d3-021b-433a-8c79-2d19cbeed6bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3784b41091646e9b77d4420f6029694'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:35.422202 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e30949d3-021b-433a-8c79-2d19cbeed6bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3784b41091646e9b77d4420f6029694'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:35.422836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e30949d3-021b-433a-8c79-2d19cbeed6bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3784b41091646e9b77d4420f6029694'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:35.429609 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e30949d3-021b-433a-8c79-2d19cbeed6bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3784b41091646e9b77d4420f6029694'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:35.497335 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1370/2720] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:35 2022] DELETE /identity/v3/users/f3784b41091646e9b77d4420f6029694 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:35.501913 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed94d13-8ffc-44e0-a649-0437e05c9939 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:35.502201 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed94d13-8ffc-44e0-a649-0437e05c9939 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:35.502514 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed94d13-8ffc-44e0-a649-0437e05c9939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:35.520280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-5ed94d13-8ffc-44e0-a649-0437e05c9939 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:35.521783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5ed94d13-8ffc-44e0-a649-0437e05c9939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:35.548915 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:35.584332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1351/2721] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:35.595088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30949d3-021b-433a-8c79-2d19cbeed6bd admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:35.596684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e30949d3-021b-433a-8c79-2d19cbeed6bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:35.663894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30949d3-021b-433a-8c79-2d19cbeed6bd admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:35.668105 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40fe7b-e41f-40e6-bb27-d2c44bf4892f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:35.668775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40fe7b-e41f-40e6-bb27-d2c44bf4892f service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:35.669060 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40fe7b-e41f-40e6-bb27-d2c44bf4892f service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:35.669336 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40fe7b-e41f-40e6-bb27-d2c44bf4892f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:35.679699 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40fe7b-e41f-40e6-bb27-d2c44bf4892f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:35.681854 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c40fe7b-e41f-40e6-bb27-d2c44bf4892f service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:35.687193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed94d13-8ffc-44e0-a649-0437e05c9939 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:35.688679 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5ed94d13-8ffc-44e0-a649-0437e05c9939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:35.692574 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1371/2722] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:35.757377 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed94d13-8ffc-44e0-a649-0437e05c9939 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:35.762784 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1fd8b-f3ad-4fa3-8b59-3f5ab1e890fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:35.763653 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1fd8b-f3ad-4fa3-8b59-3f5ab1e890fa service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:35.770556 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1fd8b-f3ad-4fa3-8b59-3f5ab1e890fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:35.770556 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1fd8b-f3ad-4fa3-8b59-3f5ab1e890fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:35.770556 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b8f1fd8b-f3ad-4fa3-8b59-3f5ab1e890fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:35.834252 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1fd8b-f3ad-4fa3-8b59-3f5ab1e890fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:35.835792 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1fd8b-f3ad-4fa3-8b59-3f5ab1e890fa service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:35.845863 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1352/2723] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:25:36.144074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40fe7b-e41f-40e6-bb27-d2c44bf4892f service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:36.145962 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6c40fe7b-e41f-40e6-bb27-d2c44bf4892f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:36.161223 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c40fe7b-e41f-40e6-bb27-d2c44bf4892f service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:36.168656 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657aad95-7a8c-4ff0-9fd8-12c79ddde07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:36.169534 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657aad95-7a8c-4ff0-9fd8-12c79ddde07a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:36.170130 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657aad95-7a8c-4ff0-9fd8-12c79ddde07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:36.170910 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657aad95-7a8c-4ff0-9fd8-12c79ddde07a admin admin] PATH_INFO: `/v3/projects/ec7dff5f75b64f6cb7109641144c552d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:36.172518 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657aad95-7a8c-4ff0-9fd8-12c79ddde07a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec7dff5f75b64f6cb7109641144c552d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:36.174523 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657aad95-7a8c-4ff0-9fd8-12c79ddde07a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:36.326560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:36.334171 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-657aad95-7a8c-4ff0-9fd8-12c79ddde07a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec7dff5f75b64f6cb7109641144c552d', 'request_id': 'req-657aad95-7a8c-4ff0-9fd8-12c79ddde07a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:36.340141 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1fd8b-f3ad-4fa3-8b59-3f5ab1e890fa service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:36.341275 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-657aad95-7a8c-4ff0-9fd8-12c79ddde07a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec7dff5f75b64f6cb7109641144c552d', 'request_id': 'req-657aad95-7a8c-4ff0-9fd8-12c79ddde07a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:36.341546 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-b8f1fd8b-f3ad-4fa3-8b59-3f5ab1e890fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:36.346768 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-657aad95-7a8c-4ff0-9fd8-12c79ddde07a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec7dff5f75b64f6cb7109641144c552d', 'request_id': 'req-657aad95-7a8c-4ff0-9fd8-12c79ddde07a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:36.353275 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1fd8b-f3ad-4fa3-8b59-3f5ab1e890fa service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:36.404845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0237902-ff87-4c79-80bb-b9469d24a255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:36.405635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0237902-ff87-4c79-80bb-b9469d24a255 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:36.406056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0237902-ff87-4c79-80bb-b9469d24a255 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:36.406311 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0237902-ff87-4c79-80bb-b9469d24a255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:36.408162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e0237902-ff87-4c79-80bb-b9469d24a255 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:36.417323 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-657aad95-7a8c-4ff0-9fd8-12c79ddde07a admin admin] The token cache is being invalidate because project ec7dff5f75b64f6cb7109641144c552d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:36.420743 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-657aad95-7a8c-4ff0-9fd8-12c79ddde07a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:36.420743 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-657aad95-7a8c-4ff0-9fd8-12c79ddde07a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:36.420743 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1372/2724] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:36 2022] DELETE /identity/v3/projects/ec7dff5f75b64f6cb7109641144c552d => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:36.423923 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657aad95-7a8c-4ff0-9fd8-12c79ddde07a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:36.425354 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-657aad95-7a8c-4ff0-9fd8-12c79ddde07a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:36.444554 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=100 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:36.475622 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0237902-ff87-4c79-80bb-b9469d24a255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:36.477259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0237902-ff87-4c79-80bb-b9469d24a255 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:36.487645 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1353/2725] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:36.499156 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657aad95-7a8c-4ff0-9fd8-12c79ddde07a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:36.502945 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84802114-83be-492a-a298-e577e0b047e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:36.503824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84802114-83be-492a-a298-e577e0b047e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:36.504108 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84802114-83be-492a-a298-e577e0b047e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:36.504397 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84802114-83be-492a-a298-e577e0b047e7 admin admin] PATH_INFO: `/v3/users/c941478608544f7799bd3cf9a7b733cd` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:36.505621 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84802114-83be-492a-a298-e577e0b047e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c941478608544f7799bd3cf9a7b733cd)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:36.508745 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84802114-83be-492a-a298-e577e0b047e7 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:36.730359 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-84802114-83be-492a-a298-e577e0b047e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c941478608544f7799bd3cf9a7b733cd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:36.749556 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-84802114-83be-492a-a298-e577e0b047e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c941478608544f7799bd3cf9a7b733cd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:36.750328 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-84802114-83be-492a-a298-e577e0b047e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c941478608544f7799bd3cf9a7b733cd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:36.751015 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-84802114-83be-492a-a298-e577e0b047e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c941478608544f7799bd3cf9a7b733cd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:36.760744 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-84802114-83be-492a-a298-e577e0b047e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c941478608544f7799bd3cf9a7b733cd'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:36.851339 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1373/2726] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:36 2022] DELETE /identity/v3/users/c941478608544f7799bd3cf9a7b733cd => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:37.281867 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0237902-ff87-4c79-80bb-b9469d24a255 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:37.295169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0237902-ff87-4c79-80bb-b9469d24a255 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:37.361692 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df309d-ed50-4d70-9a7c-61214f8c439c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:37.362589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df309d-ed50-4d70-9a7c-61214f8c439c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:37.362871 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df309d-ed50-4d70-9a7c-61214f8c439c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:37.363146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df309d-ed50-4d70-9a7c-61214f8c439c admin admin] PATH_INFO: `/v3/projects/e62cf048a94841c3b0642d5da05fcc63` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:37.364335 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df309d-ed50-4d70-9a7c-61214f8c439c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e62cf048a94841c3b0642d5da05fcc63)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:37.366283 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df309d-ed50-4d70-9a7c-61214f8c439c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:37.469046 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f6df309d-ed50-4d70-9a7c-61214f8c439c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e62cf048a94841c3b0642d5da05fcc63', 'request_id': 'req-f6df309d-ed50-4d70-9a7c-61214f8c439c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:37.559863 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f6df309d-ed50-4d70-9a7c-61214f8c439c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e62cf048a94841c3b0642d5da05fcc63', 'request_id': 'req-f6df309d-ed50-4d70-9a7c-61214f8c439c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:37.565906 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f6df309d-ed50-4d70-9a7c-61214f8c439c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e62cf048a94841c3b0642d5da05fcc63', 'request_id': 'req-f6df309d-ed50-4d70-9a7c-61214f8c439c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:37.577501 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f6df309d-ed50-4d70-9a7c-61214f8c439c admin admin] The token cache is being invalidate because project e62cf048a94841c3b0642d5da05fcc63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:37.577800 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f6df309d-ed50-4d70-9a7c-61214f8c439c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:37.578483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-f6df309d-ed50-4d70-9a7c-61214f8c439c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:37.580053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1354/2727] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:37 2022] DELETE /identity/v3/projects/e62cf048a94841c3b0642d5da05fcc63 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:37.585109 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db695bc6-84f2-4446-940b-cb8f28e62193 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:37.585582 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db695bc6-84f2-4446-940b-cb8f28e62193 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:37.587847 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db695bc6-84f2-4446-940b-cb8f28e62193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:37.604866 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-db695bc6-84f2-4446-940b-cb8f28e62193 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:37.606079 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-db695bc6-84f2-4446-940b-cb8f28e62193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:37.647774 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:37.670814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1374/2728] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:37.675139 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986fa25-b2a6-437c-be95-bbdc8599221c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:37.675430 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986fa25-b2a6-437c-be95-bbdc8599221c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:37.675743 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986fa25-b2a6-437c-be95-bbdc8599221c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:37.694710 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4986fa25-b2a6-437c-be95-bbdc8599221c None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:37.695939 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4986fa25-b2a6-437c-be95-bbdc8599221c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:37.704571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1355/2729] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:37.708503 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db695bc6-84f2-4446-940b-cb8f28e62193 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:37.719100 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db695bc6-84f2-4446-940b-cb8f28e62193 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:37.723860 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a05c5-2575-437d-979a-308738f878a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:37.724590 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a05c5-2575-437d-979a-308738f878a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:37.724931 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a05c5-2575-437d-979a-308738f878a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:37.725247 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a05c5-2575-437d-979a-308738f878a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:37.726002 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a05c5-2575-437d-979a-308738f878a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:25:37.726307 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a05c5-2575-437d-979a-308738f878a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:37.728553 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a05c5-2575-437d-979a-308738f878a8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:37.742104 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1375/2730] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:25:37.746318 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986fa25-b2a6-437c-be95-bbdc8599221c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:37.756092 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986fa25-b2a6-437c-be95-bbdc8599221c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:37.760895 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5285942-ab03-400e-9266-e60faf3431fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:37.761557 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5285942-ab03-400e-9266-e60faf3431fd admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:37.762349 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5285942-ab03-400e-9266-e60faf3431fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:37.762349 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5285942-ab03-400e-9266-e60faf3431fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:37.762912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5285942-ab03-400e-9266-e60faf3431fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:37.764821 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5285942-ab03-400e-9266-e60faf3431fd admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:37.848957 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1356/2731] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:37 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:25:37.852904 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a05c5-2575-437d-979a-308738f878a8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:37.863543 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a05c5-2575-437d-979a-308738f878a8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:37.920989 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0501d88-a483-4a06-9ebf-43b2cf562386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:37.921863 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0501d88-a483-4a06-9ebf-43b2cf562386 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:37.922147 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0501d88-a483-4a06-9ebf-43b2cf562386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:37.922451 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0501d88-a483-4a06-9ebf-43b2cf562386 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:37.923139 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0501d88-a483-4a06-9ebf-43b2cf562386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:37.925151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0501d88-a483-4a06-9ebf-43b2cf562386 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:38.061892 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1376/2732] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:37 2022] POST /identity/v3/users => generated 284 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:38.066267 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5285942-ab03-400e-9266-e60faf3431fd admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:38.078539 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5285942-ab03-400e-9266-e60faf3431fd admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:38.083120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb498731-f949-48e7-9005-86a9273fe8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:38.083781 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb498731-f949-48e7-9005-86a9273fe8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:38.084064 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb498731-f949-48e7-9005-86a9273fe8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:38.084342 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb498731-f949-48e7-9005-86a9273fe8d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:38.085064 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb498731-f949-48e7-9005-86a9273fe8d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:38.086665 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb498731-f949-48e7-9005-86a9273fe8d0 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:38.101330 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1357/2733] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:38.105874 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0501d88-a483-4a06-9ebf-43b2cf562386 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:38.116054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0501d88-a483-4a06-9ebf-43b2cf562386 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:38.131212 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc12b2-acb6-4514-82fe-a56525c22dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:38.131212 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc12b2-acb6-4514-82fe-a56525c22dab admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:38.131212 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc12b2-acb6-4514-82fe-a56525c22dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:38.131212 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc12b2-acb6-4514-82fe-a56525c22dab admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:38.131859 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc12b2-acb6-4514-82fe-a56525c22dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:38.131859 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc12b2-acb6-4514-82fe-a56525c22dab admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:38.135791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1377/2734] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:38.140331 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb498731-f949-48e7-9005-86a9273fe8d0 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:38.152644 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb498731-f949-48e7-9005-86a9273fe8d0 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:38.157166 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8f14f-ddba-409e-bfb4-0400d2849c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:38.158049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8f14f-ddba-409e-bfb4-0400d2849c89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:38.158322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8f14f-ddba-409e-bfb4-0400d2849c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:38.158635 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8f14f-ddba-409e-bfb4-0400d2849c89 admin admin] PATH_INFO: `/v3/projects/e4a2a6c9a2c94f528bf83cb33d08a748/users/f8c368a4f5d7407d9f5a517f9a50fdac/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:38.174502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8f14f-ddba-409e-bfb4-0400d2849c89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4a2a6c9a2c94f528bf83cb33d08a748, user_id=f8c368a4f5d7407d9f5a517f9a50fdac, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:38.177142 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8f14f-ddba-409e-bfb4-0400d2849c89 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:38.180180 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:38.272875 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1358/2735] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:38 2022] PUT /identity/v3/projects/e4a2a6c9a2c94f528bf83cb33d08a748/users/f8c368a4f5d7407d9f5a517f9a50fdac/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:38.283824 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfd431-3aa4-4d98-8bb1-6065d02c8de8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:38.284113 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfd431-3aa4-4d98-8bb1-6065d02c8de8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:38.284395 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfd431-3aa4-4d98-8bb1-6065d02c8de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:38.311472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-89dfd431-3aa4-4d98-8bb1-6065d02c8de8 None None] MFA Rules not processed for user `f8c368a4f5d7407d9f5a517f9a50fdac`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:38.312679 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-89dfd431-3aa4-4d98-8bb1-6065d02c8de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:38.396455 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1378/2736] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:38 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:38.401897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8f14f-ddba-409e-bfb4-0400d2849c89 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:38.471129 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8f14f-ddba-409e-bfb4-0400d2849c89 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:38.476467 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e8acdd-924e-45f6-89fe-41911790d429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:38.477133 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8acdd-924e-45f6-89fe-41911790d429 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:38.477429 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8acdd-924e-45f6-89fe-41911790d429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:38.477784 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e8acdd-924e-45f6-89fe-41911790d429 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:38.478449 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e8acdd-924e-45f6-89fe-41911790d429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:38.484494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e8acdd-924e-45f6-89fe-41911790d429 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:38.558199 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1359/2737] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:38 2022] POST /identity/v3/projects => generated 367 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:38.563089 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dfd431-3aa4-4d98-8bb1-6065d02c8de8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:38.576231 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dfd431-3aa4-4d98-8bb1-6065d02c8de8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:38.626943 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:38.645601 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246fcdd-9001-4ab0-8bb0-b33fdf924aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:38.646208 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246fcdd-9001-4ab0-8bb0-b33fdf924aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:38.646541 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246fcdd-9001-4ab0-8bb0-b33fdf924aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:38.646828 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246fcdd-9001-4ab0-8bb0-b33fdf924aa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:38.647538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246fcdd-9001-4ab0-8bb0-b33fdf924aa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:38.649690 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246fcdd-9001-4ab0-8bb0-b33fdf924aa5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:38.722330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1379/2738] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:38 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:38.726309 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e8acdd-924e-45f6-89fe-41911790d429 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:38.739082 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e8acdd-924e-45f6-89fe-41911790d429 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:38.744057 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aad43a-6110-4546-9068-1ca475b8cd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:38.744843 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aad43a-6110-4546-9068-1ca475b8cd8f admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:38.745249 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aad43a-6110-4546-9068-1ca475b8cd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:38.745532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aad43a-6110-4546-9068-1ca475b8cd8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:38.746334 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aad43a-6110-4546-9068-1ca475b8cd8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:38.747979 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aad43a-6110-4546-9068-1ca475b8cd8f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:38.762343 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1360/2739] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:38.766869 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246fcdd-9001-4ab0-8bb0-b33fdf924aa5 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:38.779774 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246fcdd-9001-4ab0-8bb0-b33fdf924aa5 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:38.786387 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4128a06-d376-491f-a5e2-760c7a77647c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:38.787737 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4128a06-d376-491f-a5e2-760c7a77647c admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:38.788025 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4128a06-d376-491f-a5e2-760c7a77647c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:38.788389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4128a06-d376-491f-a5e2-760c7a77647c admin admin] PATH_INFO: `/v3/projects/f2724a8be0ed45458769d59be6b32cd4/users/beaa573b31924db7aa9f5ead154f0d50/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:38.808427 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4128a06-d376-491f-a5e2-760c7a77647c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2724a8be0ed45458769d59be6b32cd4, user_id=beaa573b31924db7aa9f5ead154f0d50, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:38.811337 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4128a06-d376-491f-a5e2-760c7a77647c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:38.879296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1380/2740] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:38 2022] PUT /identity/v3/projects/f2724a8be0ed45458769d59be6b32cd4/users/beaa573b31924db7aa9f5ead154f0d50/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:38.883304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aad43a-6110-4546-9068-1ca475b8cd8f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:38.894422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aad43a-6110-4546-9068-1ca475b8cd8f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:38.967995 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa7ebf-6946-4d2e-b27d-b59d54cbbbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:38.968672 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa7ebf-6946-4d2e-b27d-b59d54cbbbc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:38.968952 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa7ebf-6946-4d2e-b27d-b59d54cbbbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:38.969232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa7ebf-6946-4d2e-b27d-b59d54cbbbc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:38.969955 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa7ebf-6946-4d2e-b27d-b59d54cbbbc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:38.971718 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fa7ebf-6946-4d2e-b27d-b59d54cbbbc1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:38.983801 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1361/2741] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:38.987849 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4128a06-d376-491f-a5e2-760c7a77647c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:39.000446 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4128a06-d376-491f-a5e2-760c7a77647c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:39.005167 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8a151-44b0-4561-9b17-f4118debd813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:39.006203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8a151-44b0-4561-9b17-f4118debd813 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:39.006610 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8a151-44b0-4561-9b17-f4118debd813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:39.006992 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d8a151-44b0-4561-9b17-f4118debd813 admin admin] PATH_INFO: `/v3/projects/f2724a8be0ed45458769d59be6b32cd4/users/beaa573b31924db7aa9f5ead154f0d50/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:39.008972 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8a151-44b0-4561-9b17-f4118debd813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2724a8be0ed45458769d59be6b32cd4, user_id=beaa573b31924db7aa9f5ead154f0d50, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:39.012955 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d8a151-44b0-4561-9b17-f4118debd813 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:39.024707 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1381/2742] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:38 2022] PUT /identity/v3/projects/f2724a8be0ed45458769d59be6b32cd4/users/beaa573b31924db7aa9f5ead154f0d50/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:39.038540 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efffdf6-303f-4e90-bc4b-d2dc903eabad None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:39.038907 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efffdf6-303f-4e90-bc4b-d2dc903eabad None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:39.039107 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efffdf6-303f-4e90-bc4b-d2dc903eabad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:39.069782 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9efffdf6-303f-4e90-bc4b-d2dc903eabad None None] MFA Rules not processed for user `beaa573b31924db7aa9f5ead154f0d50`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:39.071042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9efffdf6-303f-4e90-bc4b-d2dc903eabad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:39.139598 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:39.159501 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1362/2743] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:39 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:39.179229 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8a151-44b0-4561-9b17-f4118debd813 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:39.180922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-85d8a151-44b0-4561-9b17-f4118debd813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:39.248034 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d8a151-44b0-4561-9b17-f4118debd813 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:39.252079 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700b16c-fe6c-4a98-bbf1-fbbaca91d9e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:39.252745 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700b16c-fe6c-4a98-bbf1-fbbaca91d9e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:39.253032 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700b16c-fe6c-4a98-bbf1-fbbaca91d9e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:39.253451 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700b16c-fe6c-4a98-bbf1-fbbaca91d9e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:39.267304 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700b16c-fe6c-4a98-bbf1-fbbaca91d9e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:39.268753 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700b16c-fe6c-4a98-bbf1-fbbaca91d9e1 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:39.279300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1382/2744] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:39 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:39.519534 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efffdf6-303f-4e90-bc4b-d2dc903eabad None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:39.529007 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efffdf6-303f-4e90-bc4b-d2dc903eabad None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:39.532715 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be753c7e-bbb1-4709-92e6-51aed2134537 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:39.533360 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be753c7e-bbb1-4709-92e6-51aed2134537 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:39.533647 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be753c7e-bbb1-4709-92e6-51aed2134537 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:39.533927 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be753c7e-bbb1-4709-92e6-51aed2134537 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:39.590855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be753c7e-bbb1-4709-92e6-51aed2134537 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:39.592292 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be753c7e-bbb1-4709-92e6-51aed2134537 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:39.607716 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1363/2745] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:39 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:41.348761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700b16c-fe6c-4a98-bbf1-fbbaca91d9e1 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:41.350193 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-4700b16c-fe6c-4a98-bbf1-fbbaca91d9e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:41.415918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700b16c-fe6c-4a98-bbf1-fbbaca91d9e1 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:41.420590 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea7ff2-61e3-4ca5-bbcc-e4af86770e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:41.421546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea7ff2-61e3-4ca5-bbcc-e4af86770e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:41.421828 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea7ff2-61e3-4ca5-bbcc-e4af86770e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:41.422108 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ea7ff2-61e3-4ca5-bbcc-e4af86770e0b admin admin] PATH_INFO: `/v3/users/7b0c3b26ed4e4f128e780460c0bfd434` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:41.423329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea7ff2-61e3-4ca5-bbcc-e4af86770e0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b0c3b26ed4e4f128e780460c0bfd434)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:41.425252 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ea7ff2-61e3-4ca5-bbcc-e4af86770e0b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:41.609632 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:41.615318 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-85ea7ff2-61e3-4ca5-bbcc-e4af86770e0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b0c3b26ed4e4f128e780460c0bfd434'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:41.628592 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-85ea7ff2-61e3-4ca5-bbcc-e4af86770e0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b0c3b26ed4e4f128e780460c0bfd434'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:41.633962 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-85ea7ff2-61e3-4ca5-bbcc-e4af86770e0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b0c3b26ed4e4f128e780460c0bfd434'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:41.633962 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-85ea7ff2-61e3-4ca5-bbcc-e4af86770e0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b0c3b26ed4e4f128e780460c0bfd434'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:41.637141 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-85ea7ff2-61e3-4ca5-bbcc-e4af86770e0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b0c3b26ed4e4f128e780460c0bfd434'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:41.702401 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1383/2746] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:41 2022] DELETE /identity/v3/users/7b0c3b26ed4e4f128e780460c0bfd434 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:41.706967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211a963-2394-4107-9d79-6b3e36570f86 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:41.707445 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211a963-2394-4107-9d79-6b3e36570f86 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:41.707543 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211a963-2394-4107-9d79-6b3e36570f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:41.725181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-2211a963-2394-4107-9d79-6b3e36570f86 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:41.726422 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2211a963-2394-4107-9d79-6b3e36570f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:41.788832 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1364/2747] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:41.799764 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea7ff2-61e3-4ca5-bbcc-e4af86770e0b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:41.801224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-85ea7ff2-61e3-4ca5-bbcc-e4af86770e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:41.862181 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ea7ff2-61e3-4ca5-bbcc-e4af86770e0b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:41.865873 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4207bd9-e891-4088-847e-10a4b9bc65f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:41.866866 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4207bd9-e891-4088-847e-10a4b9bc65f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:41.867157 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4207bd9-e891-4088-847e-10a4b9bc65f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:41.867443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4207bd9-e891-4088-847e-10a4b9bc65f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:41.876680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4207bd9-e891-4088-847e-10a4b9bc65f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:41.878122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4207bd9-e891-4088-847e-10a4b9bc65f0 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:41.888236 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1384/2748] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:42.093287 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2211a963-2394-4107-9d79-6b3e36570f86 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:42.094857 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2211a963-2394-4107-9d79-6b3e36570f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:42.107746 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2211a963-2394-4107-9d79-6b3e36570f86 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:42.113624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94aae90-3e3c-4968-bd00-d519c7d5363f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:42.113624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94aae90-3e3c-4968-bd00-d519c7d5363f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:42.113624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94aae90-3e3c-4968-bd00-d519c7d5363f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:42.113983 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94aae90-3e3c-4968-bd00-d519c7d5363f admin admin] PATH_INFO: `/v3/users/0d08bbfc0a5a4e32bf7dcfad45c47357` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:42.115070 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94aae90-3e3c-4968-bd00-d519c7d5363f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d08bbfc0a5a4e32bf7dcfad45c47357)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:42.117005 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94aae90-3e3c-4968-bd00-d519c7d5363f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:42.352494 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4207bd9-e891-4088-847e-10a4b9bc65f0 service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:42.352494 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a4207bd9-e891-4088-847e-10a4b9bc65f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:42.365993 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4207bd9-e891-4088-847e-10a4b9bc65f0 service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:42.371441 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8fc7e-8bb8-44a9-aa60-a2956f14d206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:42.372486 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8fc7e-8bb8-44a9-aa60-a2956f14d206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:42.372775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8fc7e-8bb8-44a9-aa60-a2956f14d206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:42.373055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8fc7e-8bb8-44a9-aa60-a2956f14d206 admin admin] PATH_INFO: `/v3/projects/2a191785599744ef907f9e1f17bc572d` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:42.374283 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8fc7e-8bb8-44a9-aa60-a2956f14d206 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a191785599744ef907f9e1f17bc572d)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:42.381174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8fc7e-8bb8-44a9-aa60-a2956f14d206 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:42.387670 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c94aae90-3e3c-4968-bd00-d519c7d5363f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d08bbfc0a5a4e32bf7dcfad45c47357'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:42.402063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c94aae90-3e3c-4968-bd00-d519c7d5363f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d08bbfc0a5a4e32bf7dcfad45c47357'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:42.491428 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c94aae90-3e3c-4968-bd00-d519c7d5363f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d08bbfc0a5a4e32bf7dcfad45c47357'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:42.492042 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c94aae90-3e3c-4968-bd00-d519c7d5363f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d08bbfc0a5a4e32bf7dcfad45c47357'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:42.492646 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c94aae90-3e3c-4968-bd00-d519c7d5363f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d08bbfc0a5a4e32bf7dcfad45c47357'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:42.506196 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1365/2749] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:42 2022] DELETE /identity/v3/users/0d08bbfc0a5a4e32bf7dcfad45c47357 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:42.510933 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b48c2-48a6-4a91-9fd2-c78a9bea116c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:42.513243 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b48c2-48a6-4a91-9fd2-c78a9bea116c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:42.513243 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b48c2-48a6-4a91-9fd2-c78a9bea116c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:42.517489 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:42.520921 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-eba8fc7e-8bb8-44a9-aa60-a2956f14d206 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a191785599744ef907f9e1f17bc572d', 'request_id': 'req-eba8fc7e-8bb8-44a9-aa60-a2956f14d206'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:42.527798 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-eba8fc7e-8bb8-44a9-aa60-a2956f14d206 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a191785599744ef907f9e1f17bc572d', 'request_id': 'req-eba8fc7e-8bb8-44a9-aa60-a2956f14d206'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:42.530340 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-c65b48c2-48a6-4a91-9fd2-c78a9bea116c None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:42.531593 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c65b48c2-48a6-4a91-9fd2-c78a9bea116c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:42.534076 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-eba8fc7e-8bb8-44a9-aa60-a2956f14d206 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a191785599744ef907f9e1f17bc572d', 'request_id': 'req-eba8fc7e-8bb8-44a9-aa60-a2956f14d206'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:42.590461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1366/2750] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:42.600144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b48c2-48a6-4a91-9fd2-c78a9bea116c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:42.601511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c65b48c2-48a6-4a91-9fd2-c78a9bea116c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:42.628468 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-eba8fc7e-8bb8-44a9-aa60-a2956f14d206 admin admin] The token cache is being invalidate because project 2a191785599744ef907f9e1f17bc572d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:42.628782 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-eba8fc7e-8bb8-44a9-aa60-a2956f14d206 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:42.629538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-eba8fc7e-8bb8-44a9-aa60-a2956f14d206 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:42.631342 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1385/2751] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:42 2022] DELETE /identity/v3/projects/2a191785599744ef907f9e1f17bc572d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:42.634878 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8fc7e-8bb8-44a9-aa60-a2956f14d206 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:42.636180 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-eba8fc7e-8bb8-44a9-aa60-a2956f14d206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:42.649068 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8fc7e-8bb8-44a9-aa60-a2956f14d206 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:42.655223 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d18bce-2280-4ed6-a59e-088fb08e762b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:42.656099 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d18bce-2280-4ed6-a59e-088fb08e762b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:42.656380 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d18bce-2280-4ed6-a59e-088fb08e762b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:42.656658 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d18bce-2280-4ed6-a59e-088fb08e762b admin admin] PATH_INFO: `/v3/users/24bca992801d40da8286bd08a439a2df` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:42.657985 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d18bce-2280-4ed6-a59e-088fb08e762b admin admin] RBAC: Authorizing `identity:delete_user(user_id=24bca992801d40da8286bd08a439a2df)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:42.659689 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b48c2-48a6-4a91-9fd2-c78a9bea116c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:42.659942 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d18bce-2280-4ed6-a59e-088fb08e762b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:42.663351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab14c1c-76a5-4b3d-ab24-ebc6b3bcf69f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:42.663997 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab14c1c-76a5-4b3d-ab24-ebc6b3bcf69f service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:42.664280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab14c1c-76a5-4b3d-ab24-ebc6b3bcf69f service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:42.664564 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab14c1c-76a5-4b3d-ab24-ebc6b3bcf69f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:42.667098 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-aab14c1c-76a5-4b3d-ab24-ebc6b3bcf69f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:42.680101 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab14c1c-76a5-4b3d-ab24-ebc6b3bcf69f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:42.681531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab14c1c-76a5-4b3d-ab24-ebc6b3bcf69f service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:42.692351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1367/2752] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:25:42.874642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:42.882746 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a9d18bce-2280-4ed6-a59e-088fb08e762b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24bca992801d40da8286bd08a439a2df'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:42.894979 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a9d18bce-2280-4ed6-a59e-088fb08e762b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24bca992801d40da8286bd08a439a2df'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:42.895682 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a9d18bce-2280-4ed6-a59e-088fb08e762b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24bca992801d40da8286bd08a439a2df'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:42.896326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a9d18bce-2280-4ed6-a59e-088fb08e762b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24bca992801d40da8286bd08a439a2df'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:42.904392 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a9d18bce-2280-4ed6-a59e-088fb08e762b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24bca992801d40da8286bd08a439a2df'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:42.987742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1386/2753] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:42 2022] DELETE /identity/v3/users/24bca992801d40da8286bd08a439a2df => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:43.110068 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab14c1c-76a5-4b3d-ab24-ebc6b3bcf69f service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:43.111661 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-aab14c1c-76a5-4b3d-ab24-ebc6b3bcf69f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:43.127671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab14c1c-76a5-4b3d-ab24-ebc6b3bcf69f service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:43.190329 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53bd7b-e5e7-4f0a-a8ce-7535715d15dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:43.191228 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53bd7b-e5e7-4f0a-a8ce-7535715d15dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:43.191516 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53bd7b-e5e7-4f0a-a8ce-7535715d15dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:43.191799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53bd7b-e5e7-4f0a-a8ce-7535715d15dd admin admin] PATH_INFO: `/v3/projects/2ef415ea2ce84b9591a449dda4b59e7c` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:43.192989 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53bd7b-e5e7-4f0a-a8ce-7535715d15dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ef415ea2ce84b9591a449dda4b59e7c)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:43.194999 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53bd7b-e5e7-4f0a-a8ce-7535715d15dd admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:43.364773 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:43.366404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1d53bd7b-e5e7-4f0a-a8ce-7535715d15dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ef415ea2ce84b9591a449dda4b59e7c', 'request_id': 'req-1d53bd7b-e5e7-4f0a-a8ce-7535715d15dd'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:43.478904 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1d53bd7b-e5e7-4f0a-a8ce-7535715d15dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ef415ea2ce84b9591a449dda4b59e7c', 'request_id': 'req-1d53bd7b-e5e7-4f0a-a8ce-7535715d15dd'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:43.485806 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1d53bd7b-e5e7-4f0a-a8ce-7535715d15dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ef415ea2ce84b9591a449dda4b59e7c', 'request_id': 'req-1d53bd7b-e5e7-4f0a-a8ce-7535715d15dd'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:43.497512 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1d53bd7b-e5e7-4f0a-a8ce-7535715d15dd admin admin] The token cache is being invalidate because project 2ef415ea2ce84b9591a449dda4b59e7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:43.497808 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1d53bd7b-e5e7-4f0a-a8ce-7535715d15dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:43.498800 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1d53bd7b-e5e7-4f0a-a8ce-7535715d15dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:43.500549 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1368/2754] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:43 2022] DELETE /identity/v3/projects/2ef415ea2ce84b9591a449dda4b59e7c => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:43.504524 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d18bce-2280-4ed6-a59e-088fb08e762b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:43.509003 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a9d18bce-2280-4ed6-a59e-088fb08e762b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:43.520174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53bd7b-e5e7-4f0a-a8ce-7535715d15dd admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:43.521712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-1d53bd7b-e5e7-4f0a-a8ce-7535715d15dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:43.525311 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d18bce-2280-4ed6-a59e-088fb08e762b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:43.599495 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a18354-3442-41e1-afee-06d0fc5e4c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:43.600686 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a18354-3442-41e1-afee-06d0fc5e4c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:43.600970 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a18354-3442-41e1-afee-06d0fc5e4c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:43.601250 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a18354-3442-41e1-afee-06d0fc5e4c2c admin admin] PATH_INFO: `/v3/users/efc77d2274bf456195d03410f4eec88c` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:43.602564 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a18354-3442-41e1-afee-06d0fc5e4c2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=efc77d2274bf456195d03410f4eec88c)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:43.609216 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a18354-3442-41e1-afee-06d0fc5e4c2c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:43.609357 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53bd7b-e5e7-4f0a-a8ce-7535715d15dd admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:43.610744 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8085eb5-eae1-4946-8e74-3d053fbbb5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:43.611705 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8085eb5-eae1-4946-8e74-3d053fbbb5ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:43.611987 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8085eb5-eae1-4946-8e74-3d053fbbb5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:43.612262 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8085eb5-eae1-4946-8e74-3d053fbbb5ec admin admin] PATH_INFO: `/v3/projects/59ddee588ff346dda41ede6092d6b9c4` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:43.613454 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8085eb5-eae1-4946-8e74-3d053fbbb5ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=59ddee588ff346dda41ede6092d6b9c4)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:43.615389 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8085eb5-eae1-4946-8e74-3d053fbbb5ec admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:43.779969 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-13a18354-3442-41e1-afee-06d0fc5e4c2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efc77d2274bf456195d03410f4eec88c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:43.780193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a8085eb5-eae1-4946-8e74-3d053fbbb5ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59ddee588ff346dda41ede6092d6b9c4', 'request_id': 'req-a8085eb5-eae1-4946-8e74-3d053fbbb5ec'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:43.797949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-13a18354-3442-41e1-afee-06d0fc5e4c2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efc77d2274bf456195d03410f4eec88c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:43.799014 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-13a18354-3442-41e1-afee-06d0fc5e4c2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efc77d2274bf456195d03410f4eec88c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:43.799627 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-13a18354-3442-41e1-afee-06d0fc5e4c2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efc77d2274bf456195d03410f4eec88c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:43.807954 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-13a18354-3442-41e1-afee-06d0fc5e4c2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efc77d2274bf456195d03410f4eec88c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:43.874195 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a8085eb5-eae1-4946-8e74-3d053fbbb5ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59ddee588ff346dda41ede6092d6b9c4', 'request_id': 'req-a8085eb5-eae1-4946-8e74-3d053fbbb5ec'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:43.882187 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a8085eb5-eae1-4946-8e74-3d053fbbb5ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59ddee588ff346dda41ede6092d6b9c4', 'request_id': 'req-a8085eb5-eae1-4946-8e74-3d053fbbb5ec'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:43.893567 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a8085eb5-eae1-4946-8e74-3d053fbbb5ec admin admin] The token cache is being invalidate because project 59ddee588ff346dda41ede6092d6b9c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:43.893859 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a8085eb5-eae1-4946-8e74-3d053fbbb5ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:43.894558 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-a8085eb5-eae1-4946-8e74-3d053fbbb5ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:43.896270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1369/2755] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:43 2022] DELETE /identity/v3/projects/59ddee588ff346dda41ede6092d6b9c4 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:43.900266 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8085eb5-eae1-4946-8e74-3d053fbbb5ec admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:43.901579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a8085eb5-eae1-4946-8e74-3d053fbbb5ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:43.958913 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1387/2756] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:43 2022] DELETE /identity/v3/users/efc77d2274bf456195d03410f4eec88c => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:43.967415 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8085eb5-eae1-4946-8e74-3d053fbbb5ec admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:43.979459 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:44.040564 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627d1b4-3339-4721-8c1b-2ba052c0677a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:44.041503 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627d1b4-3339-4721-8c1b-2ba052c0677a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:44.041787 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627d1b4-3339-4721-8c1b-2ba052c0677a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:44.042063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627d1b4-3339-4721-8c1b-2ba052c0677a admin admin] PATH_INFO: `/v3/users/d9516f538b7d46549e3f0b9c0159001f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:44.043685 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627d1b4-3339-4721-8c1b-2ba052c0677a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9516f538b7d46549e3f0b9c0159001f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:44.045651 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627d1b4-3339-4721-8c1b-2ba052c0677a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:44.309855 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4627d1b4-3339-4721-8c1b-2ba052c0677a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9516f538b7d46549e3f0b9c0159001f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:44.323485 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4627d1b4-3339-4721-8c1b-2ba052c0677a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9516f538b7d46549e3f0b9c0159001f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:44.378263 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a18354-3442-41e1-afee-06d0fc5e4c2c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:44.379762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-13a18354-3442-41e1-afee-06d0fc5e4c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:44.395157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4627d1b4-3339-4721-8c1b-2ba052c0677a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9516f538b7d46549e3f0b9c0159001f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:44.395787 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4627d1b4-3339-4721-8c1b-2ba052c0677a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9516f538b7d46549e3f0b9c0159001f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:44.396410 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-4627d1b4-3339-4721-8c1b-2ba052c0677a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9516f538b7d46549e3f0b9c0159001f'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:44.400362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a18354-3442-41e1-afee-06d0fc5e4c2c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:44.405842 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca9eab-e6de-45a7-a622-55d23df98580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:44.406852 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca9eab-e6de-45a7-a622-55d23df98580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:44.407209 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca9eab-e6de-45a7-a622-55d23df98580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:44.407680 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca9eab-e6de-45a7-a622-55d23df98580 admin admin] PATH_INFO: `/v3/projects/689e29647b3d46c29d0ce35362f05f5e` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:44.409080 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca9eab-e6de-45a7-a622-55d23df98580 admin admin] RBAC: Authorizing `identity:delete_project(project_id=689e29647b3d46c29d0ce35362f05f5e)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:44.411400 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca9eab-e6de-45a7-a622-55d23df98580 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:44.412812 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1370/2757] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:43 2022] DELETE /identity/v3/users/d9516f538b7d46549e3f0b9c0159001f => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:44.518643 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:44.533869 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-82ca9eab-e6de-45a7-a622-55d23df98580 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '689e29647b3d46c29d0ce35362f05f5e', 'request_id': 'req-82ca9eab-e6de-45a7-a622-55d23df98580'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:44.539927 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-82ca9eab-e6de-45a7-a622-55d23df98580 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '689e29647b3d46c29d0ce35362f05f5e', 'request_id': 'req-82ca9eab-e6de-45a7-a622-55d23df98580'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:44.546023 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-82ca9eab-e6de-45a7-a622-55d23df98580 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '689e29647b3d46c29d0ce35362f05f5e', 'request_id': 'req-82ca9eab-e6de-45a7-a622-55d23df98580'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:44.641653 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-82ca9eab-e6de-45a7-a622-55d23df98580 admin admin] The token cache is being invalidate because project 689e29647b3d46c29d0ce35362f05f5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:44.642068 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-82ca9eab-e6de-45a7-a622-55d23df98580 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:44.642653 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-82ca9eab-e6de-45a7-a622-55d23df98580 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:44.644698 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1388/2758] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:44 2022] DELETE /identity/v3/projects/689e29647b3d46c29d0ce35362f05f5e => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:44.650650 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8690559-120d-44ef-bbc4-8ef0a3240500 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:44.651396 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8690559-120d-44ef-bbc4-8ef0a3240500 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:44.651396 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8690559-120d-44ef-bbc4-8ef0a3240500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:44.680529 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a8690559-120d-44ef-bbc4-8ef0a3240500 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:44.681763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a8690559-120d-44ef-bbc4-8ef0a3240500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:44.751406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1371/2759] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:44.759527 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2da65-6c62-48cb-94b4-f73f993c3975 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:44.759894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2da65-6c62-48cb-94b4-f73f993c3975 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:44.760246 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2da65-6c62-48cb-94b4-f73f993c3975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:44.780282 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-30b2da65-6c62-48cb-94b4-f73f993c3975 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:44.781523 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-30b2da65-6c62-48cb-94b4-f73f993c3975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:44.789731 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1389/2760] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:44.795066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8690559-120d-44ef-bbc4-8ef0a3240500 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:44.804890 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8690559-120d-44ef-bbc4-8ef0a3240500 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:44.810695 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bcf89-68b5-4fde-9b23-2582b8214607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:44.811353 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bcf89-68b5-4fde-9b23-2582b8214607 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:44.811641 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bcf89-68b5-4fde-9b23-2582b8214607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:44.811925 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bcf89-68b5-4fde-9b23-2582b8214607 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:44.812656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6bcf89-68b5-4fde-9b23-2582b8214607 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:25:44.812965 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6bcf89-68b5-4fde-9b23-2582b8214607 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:44.814723 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6bcf89-68b5-4fde-9b23-2582b8214607 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:44.826850 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1372/2761] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:25:44.831189 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2da65-6c62-48cb-94b4-f73f993c3975 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:44.846704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2da65-6c62-48cb-94b4-f73f993c3975 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:44.846704 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb41ad-21cb-47ac-808b-7a116b1e804b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:44.846932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb41ad-21cb-47ac-808b-7a116b1e804b admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:44.847189 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb41ad-21cb-47ac-808b-7a116b1e804b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:44.847572 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb41ad-21cb-47ac-808b-7a116b1e804b admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:44.848172 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb41ad-21cb-47ac-808b-7a116b1e804b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:44.850095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb41ad-21cb-47ac-808b-7a116b1e804b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:44.877735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bcf89-68b5-4fde-9b23-2582b8214607 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:44.879812 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9c6bcf89-68b5-4fde-9b23-2582b8214607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:44.896127 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bcf89-68b5-4fde-9b23-2582b8214607 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:44.908890 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbb8f5-7b2b-4619-8aff-2cb7aa59eef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:44.909757 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbb8f5-7b2b-4619-8aff-2cb7aa59eef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:44.910038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbb8f5-7b2b-4619-8aff-2cb7aa59eef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:44.910338 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbb8f5-7b2b-4619-8aff-2cb7aa59eef7 admin admin] PATH_INFO: `/v3/projects/9e69408a96c948b18c3456d4254a9930` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:44.911549 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbb8f5-7b2b-4619-8aff-2cb7aa59eef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e69408a96c948b18c3456d4254a9930)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:44.913630 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbb8f5-7b2b-4619-8aff-2cb7aa59eef7 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:44.932561 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:44.942814 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1390/2762] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:44 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:44.943970 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb41ad-21cb-47ac-808b-7a116b1e804b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:45.007257 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb41ad-21cb-47ac-808b-7a116b1e804b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:45.012382 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b63e5c-2ce1-4053-8d9a-4afff644b855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:45.013045 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b63e5c-2ce1-4053-8d9a-4afff644b855 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:45.013332 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b63e5c-2ce1-4053-8d9a-4afff644b855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:45.013609 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b63e5c-2ce1-4053-8d9a-4afff644b855 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:45.014289 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b63e5c-2ce1-4053-8d9a-4afff644b855 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:45.016229 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b63e5c-2ce1-4053-8d9a-4afff644b855 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:45.035426 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5cbbb8f5-7b2b-4619-8aff-2cb7aa59eef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e69408a96c948b18c3456d4254a9930', 'request_id': 'req-5cbbb8f5-7b2b-4619-8aff-2cb7aa59eef7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:45.126770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1391/2763] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:44 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:45.130679 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b63e5c-2ce1-4053-8d9a-4afff644b855 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:45.140693 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b63e5c-2ce1-4053-8d9a-4afff644b855 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:45.143328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5cbbb8f5-7b2b-4619-8aff-2cb7aa59eef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e69408a96c948b18c3456d4254a9930', 'request_id': 'req-5cbbb8f5-7b2b-4619-8aff-2cb7aa59eef7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:45.149577 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5cbbb8f5-7b2b-4619-8aff-2cb7aa59eef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e69408a96c948b18c3456d4254a9930', 'request_id': 'req-5cbbb8f5-7b2b-4619-8aff-2cb7aa59eef7'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:45.162719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5cbbb8f5-7b2b-4619-8aff-2cb7aa59eef7 admin admin] The token cache is being invalidate because project 9e69408a96c948b18c3456d4254a9930 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:45.162998 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5cbbb8f5-7b2b-4619-8aff-2cb7aa59eef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:45.163726 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5cbbb8f5-7b2b-4619-8aff-2cb7aa59eef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:45.165671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1373/2764] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:44 2022] DELETE /identity/v3/projects/9e69408a96c948b18c3456d4254a9930 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:45.203223 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4d398-11dc-4f62-a8fd-8b8439dd02c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:45.204062 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4d398-11dc-4f62-a8fd-8b8439dd02c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:45.204302 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4d398-11dc-4f62-a8fd-8b8439dd02c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:45.204581 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4d398-11dc-4f62-a8fd-8b8439dd02c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:45.205331 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4d398-11dc-4f62-a8fd-8b8439dd02c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:45.207102 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b4d398-11dc-4f62-a8fd-8b8439dd02c1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:45.218706 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1392/2765] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:25:45.222846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbb8f5-7b2b-4619-8aff-2cb7aa59eef7 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:45.227087 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5cbbb8f5-7b2b-4619-8aff-2cb7aa59eef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:45.238768 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbb8f5-7b2b-4619-8aff-2cb7aa59eef7 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:45.243865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6ed493-0d32-4aeb-a522-8a360f51a785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:45.244644 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ed493-0d32-4aeb-a522-8a360f51a785 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:45.244922 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ed493-0d32-4aeb-a522-8a360f51a785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:45.245204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ed493-0d32-4aeb-a522-8a360f51a785 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:45.246518 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ed493-0d32-4aeb-a522-8a360f51a785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:45.248328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6ed493-0d32-4aeb-a522-8a360f51a785 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:45.259608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1374/2766] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:45.265453 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4d398-11dc-4f62-a8fd-8b8439dd02c1 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:45.276459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4d398-11dc-4f62-a8fd-8b8439dd02c1 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:45.281883 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad67853f-df8b-4f87-bfda-5c419131a768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:45.282829 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67853f-df8b-4f87-bfda-5c419131a768 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:45.286784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67853f-df8b-4f87-bfda-5c419131a768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:45.286784 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad67853f-df8b-4f87-bfda-5c419131a768 admin admin] PATH_INFO: `/v3/projects/f82e6522da024b158bceaa172bec65ac/users/cb95c00acc134084a4dd3f74dfcdaa3a/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:45.300500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad67853f-df8b-4f87-bfda-5c419131a768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f82e6522da024b158bceaa172bec65ac, user_id=cb95c00acc134084a4dd3f74dfcdaa3a, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:45.303367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad67853f-df8b-4f87-bfda-5c419131a768 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:45.366030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:45.394915 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1393/2767] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:45 2022] PUT /identity/v3/projects/f82e6522da024b158bceaa172bec65ac/users/cb95c00acc134084a4dd3f74dfcdaa3a/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:25:45.408428 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5af2c-cc75-4f2e-8bd8-a827b20215b0 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:45.408926 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5af2c-cc75-4f2e-8bd8-a827b20215b0 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:45.409117 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5af2c-cc75-4f2e-8bd8-a827b20215b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:45.437466 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-92b5af2c-cc75-4f2e-8bd8-a827b20215b0 None None] MFA Rules not processed for user `cb95c00acc134084a4dd3f74dfcdaa3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:45.438719 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-92b5af2c-cc75-4f2e-8bd8-a827b20215b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:45.503552 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1375/2768] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:45.515363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad67853f-df8b-4f87-bfda-5c419131a768 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:45.517044 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ad67853f-df8b-4f87-bfda-5c419131a768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:45.576269 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad67853f-df8b-4f87-bfda-5c419131a768 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:45.580054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36481f62-c6f7-472e-8b73-2af6460e45d4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61459e15d327405b88a2fdfc1fc0fecd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:45.580702 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36481f62-c6f7-472e-8b73-2af6460e45d4 service glance] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:45.580990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36481f62-c6f7-472e-8b73-2af6460e45d4 service glance] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:45.581273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36481f62-c6f7-472e-8b73-2af6460e45d4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:45.589798 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36481f62-c6f7-472e-8b73-2af6460e45d4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:45.591520 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36481f62-c6f7-472e-8b73-2af6460e45d4 service glance] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:45.600333 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1394/2769] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:46.278881 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5af2c-cc75-4f2e-8bd8-a827b20215b0 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:46.280289 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-92b5af2c-cc75-4f2e-8bd8-a827b20215b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:46.345765 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5af2c-cc75-4f2e-8bd8-a827b20215b0 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:46.350340 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843626ba-4096-42e2-b5a3-d7d40aab49aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:46.351231 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843626ba-4096-42e2-b5a3-d7d40aab49aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:46.351515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843626ba-4096-42e2-b5a3-d7d40aab49aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:46.351795 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843626ba-4096-42e2-b5a3-d7d40aab49aa admin admin] PATH_INFO: `/v3/users/f8c368a4f5d7407d9f5a517f9a50fdac` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:46.352993 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843626ba-4096-42e2-b5a3-d7d40aab49aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8c368a4f5d7407d9f5a517f9a50fdac)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:46.354961 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843626ba-4096-42e2-b5a3-d7d40aab49aa admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:46.376878 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:46.585496 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-843626ba-4096-42e2-b5a3-d7d40aab49aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8c368a4f5d7407d9f5a517f9a50fdac'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:46.597135 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-843626ba-4096-42e2-b5a3-d7d40aab49aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8c368a4f5d7407d9f5a517f9a50fdac'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:46.668556 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-843626ba-4096-42e2-b5a3-d7d40aab49aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8c368a4f5d7407d9f5a517f9a50fdac'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:46.669199 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-843626ba-4096-42e2-b5a3-d7d40aab49aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8c368a4f5d7407d9f5a517f9a50fdac'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:46.669796 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-843626ba-4096-42e2-b5a3-d7d40aab49aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8c368a4f5d7407d9f5a517f9a50fdac'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:46.684483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1376/2770] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:46 2022] DELETE /identity/v3/users/f8c368a4f5d7407d9f5a517f9a50fdac => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:46.688862 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9de77f-f27e-42f3-98c9-364c516e35bb None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:46.689363 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9de77f-f27e-42f3-98c9-364c516e35bb None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:46.689602 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9de77f-f27e-42f3-98c9-364c516e35bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:46.708924 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6a9de77f-f27e-42f3-98c9-364c516e35bb None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:46.710116 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6a9de77f-f27e-42f3-98c9-364c516e35bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:46.769918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1395/2771] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:46.780527 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843626ba-4096-42e2-b5a3-d7d40aab49aa admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:46.782074 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-843626ba-4096-42e2-b5a3-d7d40aab49aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:46.841937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843626ba-4096-42e2-b5a3-d7d40aab49aa admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:46.845817 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43b6b2-69d2-4e21-831b-735653a48b78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:46.846493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43b6b2-69d2-4e21-831b-735653a48b78 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:46.846775 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43b6b2-69d2-4e21-831b-735653a48b78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:46.847049 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43b6b2-69d2-4e21-831b-735653a48b78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:46.856442 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43b6b2-69d2-4e21-831b-735653a48b78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:46.857864 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43b6b2-69d2-4e21-831b-735653a48b78 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:46.869633 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1377/2772] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:47.366472 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9de77f-f27e-42f3-98c9-364c516e35bb None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:47.374779 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6a9de77f-f27e-42f3-98c9-364c516e35bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:47.381037 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9de77f-f27e-42f3-98c9-364c516e35bb None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:47.385920 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1b93a-599e-4461-92a1-53992745fba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:47.387390 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1b93a-599e-4461-92a1-53992745fba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:47.387876 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1b93a-599e-4461-92a1-53992745fba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:47.388151 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1b93a-599e-4461-92a1-53992745fba0 admin admin] PATH_INFO: `/v3/projects/e4a2a6c9a2c94f528bf83cb33d08a748` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:47.389374 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1b93a-599e-4461-92a1-53992745fba0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4a2a6c9a2c94f528bf83cb33d08a748)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:47.391384 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1b93a-599e-4461-92a1-53992745fba0 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:47.501196 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-55a1b93a-599e-4461-92a1-53992745fba0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4a2a6c9a2c94f528bf83cb33d08a748', 'request_id': 'req-55a1b93a-599e-4461-92a1-53992745fba0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:47.510141 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-55a1b93a-599e-4461-92a1-53992745fba0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4a2a6c9a2c94f528bf83cb33d08a748', 'request_id': 'req-55a1b93a-599e-4461-92a1-53992745fba0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:47.517214 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-55a1b93a-599e-4461-92a1-53992745fba0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4a2a6c9a2c94f528bf83cb33d08a748', 'request_id': 'req-55a1b93a-599e-4461-92a1-53992745fba0'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:47.588586 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-55a1b93a-599e-4461-92a1-53992745fba0 admin admin] The token cache is being invalidate because project e4a2a6c9a2c94f528bf83cb33d08a748 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:47.588877 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-55a1b93a-599e-4461-92a1-53992745fba0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:47.589638 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-55a1b93a-599e-4461-92a1-53992745fba0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:47.591287 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1396/2773] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:47 2022] DELETE /identity/v3/projects/e4a2a6c9a2c94f528bf83cb33d08a748 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:47.595367 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43b6b2-69d2-4e21-831b-735653a48b78 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:47.607631 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43b6b2-69d2-4e21-831b-735653a48b78 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:47.668329 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72fdf2-1b0b-49c1-a64f-cccaf0836623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:47.669209 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72fdf2-1b0b-49c1-a64f-cccaf0836623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:47.669500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72fdf2-1b0b-49c1-a64f-cccaf0836623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:47.669776 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72fdf2-1b0b-49c1-a64f-cccaf0836623 admin admin] PATH_INFO: `/v3/users/beaa573b31924db7aa9f5ead154f0d50` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:47.671010 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72fdf2-1b0b-49c1-a64f-cccaf0836623 admin admin] RBAC: Authorizing `identity:delete_user(user_id=beaa573b31924db7aa9f5ead154f0d50)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:47.673041 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72fdf2-1b0b-49c1-a64f-cccaf0836623 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:47.891636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:47.893345 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1a72fdf2-1b0b-49c1-a64f-cccaf0836623 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beaa573b31924db7aa9f5ead154f0d50'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:47.907532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1a72fdf2-1b0b-49c1-a64f-cccaf0836623 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beaa573b31924db7aa9f5ead154f0d50'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:48.023304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1a72fdf2-1b0b-49c1-a64f-cccaf0836623 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beaa573b31924db7aa9f5ead154f0d50'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:48.023966 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1a72fdf2-1b0b-49c1-a64f-cccaf0836623 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beaa573b31924db7aa9f5ead154f0d50'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:48.025564 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-1a72fdf2-1b0b-49c1-a64f-cccaf0836623 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beaa573b31924db7aa9f5ead154f0d50'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:48.047300 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1378/2774] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:25:47 2022] DELETE /identity/v3/users/beaa573b31924db7aa9f5ead154f0d50 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:48.516538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1b93a-599e-4461-92a1-53992745fba0 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:48.518081 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-55a1b93a-599e-4461-92a1-53992745fba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:48.533828 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1b93a-599e-4461-92a1-53992745fba0 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:48.550208 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:48.606775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59375d3-ef74-4506-8f7a-3da7d7639ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:48.607677 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59375d3-ef74-4506-8f7a-3da7d7639ca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:48.607968 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59375d3-ef74-4506-8f7a-3da7d7639ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:48.608252 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59375d3-ef74-4506-8f7a-3da7d7639ca1 admin admin] PATH_INFO: `/v3/projects/f2724a8be0ed45458769d59be6b32cd4` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:48.609455 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59375d3-ef74-4506-8f7a-3da7d7639ca1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2724a8be0ed45458769d59be6b32cd4)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:48.611429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59375d3-ef74-4506-8f7a-3da7d7639ca1 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:48.734663 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a59375d3-ef74-4506-8f7a-3da7d7639ca1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2724a8be0ed45458769d59be6b32cd4', 'request_id': 'req-a59375d3-ef74-4506-8f7a-3da7d7639ca1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:48.742273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a59375d3-ef74-4506-8f7a-3da7d7639ca1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2724a8be0ed45458769d59be6b32cd4', 'request_id': 'req-a59375d3-ef74-4506-8f7a-3da7d7639ca1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:48.749293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a59375d3-ef74-4506-8f7a-3da7d7639ca1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2724a8be0ed45458769d59be6b32cd4', 'request_id': 'req-a59375d3-ef74-4506-8f7a-3da7d7639ca1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:48.949788 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a59375d3-ef74-4506-8f7a-3da7d7639ca1 admin admin] The token cache is being invalidate because project f2724a8be0ed45458769d59be6b32cd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:25:48.950126 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a59375d3-ef74-4506-8f7a-3da7d7639ca1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:48.950761 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-a59375d3-ef74-4506-8f7a-3da7d7639ca1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:25:48.952308 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1397/2775] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:25:48 2022] DELETE /identity/v3/projects/f2724a8be0ed45458769d59be6b32cd4 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:48.957348 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705a98d-ecff-4744-a879-aaf75684de49 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:48.957845 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705a98d-ecff-4744-a879-aaf75684de49 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:48.957954 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705a98d-ecff-4744-a879-aaf75684de49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:48.975913 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-4705a98d-ecff-4744-a879-aaf75684de49 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:48.977137 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-4705a98d-ecff-4744-a879-aaf75684de49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:49.044532 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1379/2776] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:49.049158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2374407-4ca2-4c7d-8faf-3ce0f8158387 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:49.049447 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2374407-4ca2-4c7d-8faf-3ce0f8158387 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:49.049725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2374407-4ca2-4c7d-8faf-3ce0f8158387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:49.071346 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-f2374407-4ca2-4c7d-8faf-3ce0f8158387 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:49.072567 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f2374407-4ca2-4c7d-8faf-3ce0f8158387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:49.081108 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1398/2777] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:49.085364 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705a98d-ecff-4744-a879-aaf75684de49 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:49.095808 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705a98d-ecff-4744-a879-aaf75684de49 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:49.100609 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e4487-fa52-4145-ad88-ba8af63c1192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:49.101285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e4487-fa52-4145-ad88-ba8af63c1192 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:49.101571 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e4487-fa52-4145-ad88-ba8af63c1192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:49.101854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e4487-fa52-4145-ad88-ba8af63c1192 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:49.102625 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e4487-fa52-4145-ad88-ba8af63c1192 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:25:49.102940 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e4487-fa52-4145-ad88-ba8af63c1192 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:49.104549 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e4487-fa52-4145-ad88-ba8af63c1192 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:49.117547 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1380/2778] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:25:49.124117 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2374407-4ca2-4c7d-8faf-3ce0f8158387 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:49.135520 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2374407-4ca2-4c7d-8faf-3ce0f8158387 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:49.140901 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a1527-df8b-406a-bc52-1fba5b4a9598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:49.141569 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a1527-df8b-406a-bc52-1fba5b4a9598 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:49.141853 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a1527-df8b-406a-bc52-1fba5b4a9598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:49.142136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a1527-df8b-406a-bc52-1fba5b4a9598 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:49.142876 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a1527-df8b-406a-bc52-1fba5b4a9598 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:49.145141 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a1527-df8b-406a-bc52-1fba5b4a9598 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:49.212888 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1399/2779] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:49 2022] POST /identity/v3/projects => generated 371 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:49.221423 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e4487-fa52-4145-ad88-ba8af63c1192 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:49.232806 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e4487-fa52-4145-ad88-ba8af63c1192 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:49.291323 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfe7e3-4ee8-41d5-a860-7f52ae75ac6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:49.291992 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfe7e3-4ee8-41d5-a860-7f52ae75ac6b admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:49.292275 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfe7e3-4ee8-41d5-a860-7f52ae75ac6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:49.292558 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bfe7e3-4ee8-41d5-a860-7f52ae75ac6b admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:49.293249 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfe7e3-4ee8-41d5-a860-7f52ae75ac6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:49.296646 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bfe7e3-4ee8-41d5-a860-7f52ae75ac6b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:49.389084 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1381/2780] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:49 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:49.393429 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a1527-df8b-406a-bc52-1fba5b4a9598 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:49.404686 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a1527-df8b-406a-bc52-1fba5b4a9598 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:49.409478 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d016b-f693-4b75-b39a-291fc44155c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:49.410145 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d016b-f693-4b75-b39a-291fc44155c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:49.410467 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d016b-f693-4b75-b39a-291fc44155c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:49.410751 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d016b-f693-4b75-b39a-291fc44155c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:49.411483 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d016b-f693-4b75-b39a-291fc44155c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:49.413087 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d016b-f693-4b75-b39a-291fc44155c4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:49.426299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1400/2781] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:49.430415 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfe7e3-4ee8-41d5-a860-7f52ae75ac6b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:49.441056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bfe7e3-4ee8-41d5-a860-7f52ae75ac6b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:49.445712 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e440c-610c-48d8-b3ae-0f649b14f805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:49.446391 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e440c-610c-48d8-b3ae-0f649b14f805 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:49.446693 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e440c-610c-48d8-b3ae-0f649b14f805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:49.446973 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e440c-610c-48d8-b3ae-0f649b14f805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:49.447717 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e440c-610c-48d8-b3ae-0f649b14f805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:49.449296 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90e440c-610c-48d8-b3ae-0f649b14f805 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:49.459729 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1382/2782] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:25:49.463909 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d016b-f693-4b75-b39a-291fc44155c4 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:49.474978 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d016b-f693-4b75-b39a-291fc44155c4 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:49.479997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e3258-c792-4e0c-9908-a4c53ca6cccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:49.480740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e3258-c792-4e0c-9908-a4c53ca6cccc admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:49.481009 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e3258-c792-4e0c-9908-a4c53ca6cccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:49.481282 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8e3258-c792-4e0c-9908-a4c53ca6cccc admin admin] PATH_INFO: `/v3/projects/b3bcb19357c24ccba28c50654673d103/users/8ec358de5080410bb5d578bd59df67db/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:49.498549 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e3258-c792-4e0c-9908-a4c53ca6cccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3bcb19357c24ccba28c50654673d103, user_id=8ec358de5080410bb5d578bd59df67db, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:49.501216 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8e3258-c792-4e0c-9908-a4c53ca6cccc admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:49.572048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1401/2783] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:49 2022] PUT /identity/v3/projects/b3bcb19357c24ccba28c50654673d103/users/8ec358de5080410bb5d578bd59df67db/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:49.577965 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22ae87-8ef2-45ef-b095-20434dfc4509 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:49.578282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22ae87-8ef2-45ef-b095-20434dfc4509 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:49.578627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22ae87-8ef2-45ef-b095-20434dfc4509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:49.598435 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-dd22ae87-8ef2-45ef-b095-20434dfc4509 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:49.599704 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-dd22ae87-8ef2-45ef-b095-20434dfc4509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:49.627892 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:49.668174 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1383/2784] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:25:49.676360 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e3258-c792-4e0c-9908-a4c53ca6cccc admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:49.681763 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-ab8e3258-c792-4e0c-9908-a4c53ca6cccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:49.726719 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:49.738929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8e3258-c792-4e0c-9908-a4c53ca6cccc admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:49.743771 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b76e63-8c0a-42a0-9967-3aa5b247dd5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:49.743771 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b76e63-8c0a-42a0-9967-3aa5b247dd5e service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:49.743771 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b76e63-8c0a-42a0-9967-3aa5b247dd5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:49.744127 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b76e63-8c0a-42a0-9967-3aa5b247dd5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:49.755041 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b76e63-8c0a-42a0-9967-3aa5b247dd5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:49.756524 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b76e63-8c0a-42a0-9967-3aa5b247dd5e service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:49.767312 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1402/2785] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:51.055797 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f1cc9-361e-4b55-93d4-27b3ba593440 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:51.055797 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f1cc9-361e-4b55-93d4-27b3ba593440 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:51.055797 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f1cc9-361e-4b55-93d4-27b3ba593440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:51.086379 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f69f1cc9-361e-4b55-93d4-27b3ba593440 None None] MFA Rules not processed for user `8ec358de5080410bb5d578bd59df67db`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:51.092499 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f69f1cc9-361e-4b55-93d4-27b3ba593440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:51.165121 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1384/2786] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:51.170511 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b76e63-8c0a-42a0-9967-3aa5b247dd5e service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:51.180296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b76e63-8c0a-42a0-9967-3aa5b247dd5e service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:51.185554 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa5f9f-5fd6-427d-ba6d-5877b9a66826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:51.186326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa5f9f-5fd6-427d-ba6d-5877b9a66826 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:51.186740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa5f9f-5fd6-427d-ba6d-5877b9a66826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:51.187079 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa5f9f-5fd6-427d-ba6d-5877b9a66826 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:51.187803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa5f9f-5fd6-427d-ba6d-5877b9a66826 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:51.189720 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa5f9f-5fd6-427d-ba6d-5877b9a66826 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:51.293115 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1403/2787] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:25:51 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:51.298215 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f1cc9-361e-4b55-93d4-27b3ba593440 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:51.309181 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f1cc9-361e-4b55-93d4-27b3ba593440 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:51.366476 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdfa1c-726a-4ce6-8b76-626d56104d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:51.367157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdfa1c-726a-4ce6-8b76-626d56104d3a admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:51.367447 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdfa1c-726a-4ce6-8b76-626d56104d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:51.367727 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cdfa1c-726a-4ce6-8b76-626d56104d3a admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:51.368445 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdfa1c-726a-4ce6-8b76-626d56104d3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:51.370406 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cdfa1c-726a-4ce6-8b76-626d56104d3a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:51.491971 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1385/2788] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:25:51 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:25:51.495963 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa5f9f-5fd6-427d-ba6d-5877b9a66826 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:51.506433 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa5f9f-5fd6-427d-ba6d-5877b9a66826 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:51.511087 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391554fc-52ae-4976-ae32-7cba57a98caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:51.511742 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391554fc-52ae-4976-ae32-7cba57a98caa admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:51.512032 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391554fc-52ae-4976-ae32-7cba57a98caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:51.512314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391554fc-52ae-4976-ae32-7cba57a98caa admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:51.513054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391554fc-52ae-4976-ae32-7cba57a98caa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:51.515246 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391554fc-52ae-4976-ae32-7cba57a98caa admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:51.526500 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1404/2789] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:51.531502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdfa1c-726a-4ce6-8b76-626d56104d3a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:51.542587 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cdfa1c-726a-4ce6-8b76-626d56104d3a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:51.547519 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce7110-4369-411c-bbaa-5af33170981b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:51.548407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce7110-4369-411c-bbaa-5af33170981b admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:51.548696 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce7110-4369-411c-bbaa-5af33170981b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:51.549007 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce7110-4369-411c-bbaa-5af33170981b admin admin] PATH_INFO: `/v3/projects/b61d9a36f41e4f7d9967cc973e0864d9/users/3505f9823bf4469293523eee1d4638ff/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:51.564921 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce7110-4369-411c-bbaa-5af33170981b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b61d9a36f41e4f7d9967cc973e0864d9, user_id=3505f9823bf4469293523eee1d4638ff, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:51.567682 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce7110-4369-411c-bbaa-5af33170981b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:51.658153 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1386/2790] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:51 2022] PUT /identity/v3/projects/b61d9a36f41e4f7d9967cc973e0864d9/users/3505f9823bf4469293523eee1d4638ff/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:51.662775 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391554fc-52ae-4976-ae32-7cba57a98caa admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:51.673273 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391554fc-52ae-4976-ae32-7cba57a98caa admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:51.732218 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b123e7-5d9d-4df1-b074-ca76f76904c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:51.732922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b123e7-5d9d-4df1-b074-ca76f76904c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:51.733213 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b123e7-5d9d-4df1-b074-ca76f76904c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:51.733492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b123e7-5d9d-4df1-b074-ca76f76904c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:51.734261 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b123e7-5d9d-4df1-b074-ca76f76904c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:51.735939 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b123e7-5d9d-4df1-b074-ca76f76904c6 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:51.746248 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1405/2791] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:25:51.751071 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce7110-4369-411c-bbaa-5af33170981b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:51.762279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce7110-4369-411c-bbaa-5af33170981b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:51.767128 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a2fd9-9ac2-455e-8127-c387ae81ea92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:51.768025 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a2fd9-9ac2-455e-8127-c387ae81ea92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:51.768312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a2fd9-9ac2-455e-8127-c387ae81ea92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:51.768619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008a2fd9-9ac2-455e-8127-c387ae81ea92 admin admin] PATH_INFO: `/v3/projects/b61d9a36f41e4f7d9967cc973e0864d9/users/3505f9823bf4469293523eee1d4638ff/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:51.770628 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a2fd9-9ac2-455e-8127-c387ae81ea92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b61d9a36f41e4f7d9967cc973e0864d9, user_id=3505f9823bf4469293523eee1d4638ff, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:51.773279 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008a2fd9-9ac2-455e-8127-c387ae81ea92 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:51.785399 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1387/2792] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:25:51 2022] PUT /identity/v3/projects/b61d9a36f41e4f7d9967cc973e0864d9/users/3505f9823bf4469293523eee1d4638ff/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:25:53.102033 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89654699-726d-4331-b33e-d4de9c322a57 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:53.103915 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89654699-726d-4331-b33e-d4de9c322a57 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:53.103915 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89654699-726d-4331-b33e-d4de9c322a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:53.135336 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-89654699-726d-4331-b33e-d4de9c322a57 None None] MFA Rules not processed for user `3505f9823bf4469293523eee1d4638ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:25:53.137068 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-89654699-726d-4331-b33e-d4de9c322a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:53.215597 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1406/2793] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:25:53 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:25:53.227326 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a2fd9-9ac2-455e-8127-c387ae81ea92 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:53.229099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-008a2fd9-9ac2-455e-8127-c387ae81ea92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:25:53.244167 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:25:53.293489 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008a2fd9-9ac2-455e-8127-c387ae81ea92 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:53.297336 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f44675-b8f1-4559-9d74-5a1da49ca001 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:53.301502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f44675-b8f1-4559-9d74-5a1da49ca001 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:53.301502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f44675-b8f1-4559-9d74-5a1da49ca001 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:53.301502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f44675-b8f1-4559-9d74-5a1da49ca001 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:53.356638 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f44675-b8f1-4559-9d74-5a1da49ca001 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:53.358363 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f44675-b8f1-4559-9d74-5a1da49ca001 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:53.367339 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1388/2794] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:25:55.101911 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89654699-726d-4331-b33e-d4de9c322a57 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:25:55.112903 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89654699-726d-4331-b33e-d4de9c322a57 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:25:55.116667 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7208db4-f7f1-4459-8ab4-504ebcbf01ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:25:55.117329 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7208db4-f7f1-4459-8ab4-504ebcbf01ac service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:25:55.117617 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7208db4-f7f1-4459-8ab4-504ebcbf01ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:25:55.117898 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7208db4-f7f1-4459-8ab4-504ebcbf01ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:25:55.128486 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7208db4-f7f1-4459-8ab4-504ebcbf01ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:25:55.129932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7208db4-f7f1-4459-8ab4-504ebcbf01ac service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:25:55.140047 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1407/2795] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:25:55 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:26:03.363227 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=43 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:04.943922 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f44675-b8f1-4559-9d74-5a1da49ca001 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:04.945381 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-67f44675-b8f1-4559-9d74-5a1da49ca001 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:05.012461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f44675-b8f1-4559-9d74-5a1da49ca001 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:05.017062 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3146ef9-216e-42e4-8ca0-eaf8be8cdf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:05.017974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3146ef9-216e-42e4-8ca0-eaf8be8cdf7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:05.018280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3146ef9-216e-42e4-8ca0-eaf8be8cdf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:05.018586 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3146ef9-216e-42e4-8ca0-eaf8be8cdf7f admin admin] PATH_INFO: `/v3/users/cb95c00acc134084a4dd3f74dfcdaa3a` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:05.019832 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3146ef9-216e-42e4-8ca0-eaf8be8cdf7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb95c00acc134084a4dd3f74dfcdaa3a)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:05.021928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3146ef9-216e-42e4-8ca0-eaf8be8cdf7f admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:05.135501 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:05.230924 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c3146ef9-216e-42e4-8ca0-eaf8be8cdf7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb95c00acc134084a4dd3f74dfcdaa3a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:05.242694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c3146ef9-216e-42e4-8ca0-eaf8be8cdf7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb95c00acc134084a4dd3f74dfcdaa3a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:05.348322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c3146ef9-216e-42e4-8ca0-eaf8be8cdf7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb95c00acc134084a4dd3f74dfcdaa3a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:05.348873 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c3146ef9-216e-42e4-8ca0-eaf8be8cdf7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb95c00acc134084a4dd3f74dfcdaa3a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:05.349495 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-c3146ef9-216e-42e4-8ca0-eaf8be8cdf7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb95c00acc134084a4dd3f74dfcdaa3a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:05.362229 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1389/2796] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:26:04 2022] DELETE /identity/v3/users/cb95c00acc134084a4dd3f74dfcdaa3a => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:26:05.367090 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322c060-2692-4b5f-803b-a998b63b7b9a None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:05.367451 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322c060-2692-4b5f-803b-a998b63b7b9a None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:05.367805 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322c060-2692-4b5f-803b-a998b63b7b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:05.385776 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-9322c060-2692-4b5f-803b-a998b63b7b9a None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:05.387009 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9322c060-2692-4b5f-803b-a998b63b7b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:05.446874 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1408/2797] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:26:05.456808 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3146ef9-216e-42e4-8ca0-eaf8be8cdf7f admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:05.458224 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-c3146ef9-216e-42e4-8ca0-eaf8be8cdf7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:05.517589 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3146ef9-216e-42e4-8ca0-eaf8be8cdf7f admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:05.521412 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90821412-58eb-4312-9919-7d10734eec71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:05.522065 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90821412-58eb-4312-9919-7d10734eec71 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:05.522351 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90821412-58eb-4312-9919-7d10734eec71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:05.522663 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90821412-58eb-4312-9919-7d10734eec71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:05.532202 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90821412-58eb-4312-9919-7d10734eec71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:05.533624 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90821412-58eb-4312-9919-7d10734eec71 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:05.543940 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1390/2798] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:26:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:26:05.932521 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322c060-2692-4b5f-803b-a998b63b7b9a None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:05.934017 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9322c060-2692-4b5f-803b-a998b63b7b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:05.948692 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322c060-2692-4b5f-803b-a998b63b7b9a None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:05.953740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7a973-897f-4dda-a9d5-7e2c0d445a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:05.954655 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7a973-897f-4dda-a9d5-7e2c0d445a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:05.954946 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7a973-897f-4dda-a9d5-7e2c0d445a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:05.955221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7a973-897f-4dda-a9d5-7e2c0d445a2e admin admin] PATH_INFO: `/v3/projects/f82e6522da024b158bceaa172bec65ac` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:05.956444 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7a973-897f-4dda-a9d5-7e2c0d445a2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f82e6522da024b158bceaa172bec65ac)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:05.959732 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7a973-897f-4dda-a9d5-7e2c0d445a2e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:06.091864 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-06a7a973-897f-4dda-a9d5-7e2c0d445a2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f82e6522da024b158bceaa172bec65ac', 'request_id': 'req-06a7a973-897f-4dda-a9d5-7e2c0d445a2e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:06.098223 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-06a7a973-897f-4dda-a9d5-7e2c0d445a2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f82e6522da024b158bceaa172bec65ac', 'request_id': 'req-06a7a973-897f-4dda-a9d5-7e2c0d445a2e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:06.104056 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-06a7a973-897f-4dda-a9d5-7e2c0d445a2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f82e6522da024b158bceaa172bec65ac', 'request_id': 'req-06a7a973-897f-4dda-a9d5-7e2c0d445a2e'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:06.181695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-06a7a973-897f-4dda-a9d5-7e2c0d445a2e admin admin] The token cache is being invalidate because project f82e6522da024b158bceaa172bec65ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:26:06.182281 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-06a7a973-897f-4dda-a9d5-7e2c0d445a2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:06.182642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-06a7a973-897f-4dda-a9d5-7e2c0d445a2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:06.184188 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1409/2799] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:26:05 2022] DELETE /identity/v3/projects/f82e6522da024b158bceaa172bec65ac => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:26:06.189097 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266061b0-f5ec-4b26-8b2a-9694ef870387 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:06.189542 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266061b0-f5ec-4b26-8b2a-9694ef870387 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:06.189757 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266061b0-f5ec-4b26-8b2a-9694ef870387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:06.207988 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-266061b0-f5ec-4b26-8b2a-9694ef870387 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:06.211591 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-266061b0-f5ec-4b26-8b2a-9694ef870387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:06.228207 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:06.273468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1391/2800] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:26:06.285434 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539a29b-a975-4cf7-a1af-c8000fc80de7 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:06.285714 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539a29b-a975-4cf7-a1af-c8000fc80de7 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:06.285997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539a29b-a975-4cf7-a1af-c8000fc80de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:06.305272 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-7539a29b-a975-4cf7-a1af-c8000fc80de7 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:06.306561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7539a29b-a975-4cf7-a1af-c8000fc80de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:06.314960 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1410/2801] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:26:06.320038 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266061b0-f5ec-4b26-8b2a-9694ef870387 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:06.329959 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266061b0-f5ec-4b26-8b2a-9694ef870387 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:06.335241 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a766a-7dae-4efc-8b42-7245cb120722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:06.335888 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a766a-7dae-4efc-8b42-7245cb120722 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:06.336171 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a766a-7dae-4efc-8b42-7245cb120722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:06.336446 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a766a-7dae-4efc-8b42-7245cb120722 admin admin] PATH_INFO: `/v3/domains` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:06.337166 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a766a-7dae-4efc-8b42-7245cb120722 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80600) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:26:06.337478 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a766a-7dae-4efc-8b42-7245cb120722 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:06.339327 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5a766a-7dae-4efc-8b42-7245cb120722 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:06.351396 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1392/2802] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:26:06.355724 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539a29b-a975-4cf7-a1af-c8000fc80de7 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:06.366275 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539a29b-a975-4cf7-a1af-c8000fc80de7 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:06.371173 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf206d3c-e1ca-4698-aaca-b021cd53b554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:06.372090 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf206d3c-e1ca-4698-aaca-b021cd53b554 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:06.372332 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf206d3c-e1ca-4698-aaca-b021cd53b554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:06.372619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf206d3c-e1ca-4698-aaca-b021cd53b554 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:06.373330 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf206d3c-e1ca-4698-aaca-b021cd53b554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:06.375376 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf206d3c-e1ca-4698-aaca-b021cd53b554 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:06.467928 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1411/2803] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:26:06 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:26:06.469094 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a766a-7dae-4efc-8b42-7245cb120722 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:06.479416 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a766a-7dae-4efc-8b42-7245cb120722 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:06.537118 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443fcac8-5b52-4378-8ce1-ee0dd6057665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:06.537784 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fcac8-5b52-4378-8ce1-ee0dd6057665 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:06.538069 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fcac8-5b52-4378-8ce1-ee0dd6057665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:06.538344 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fcac8-5b52-4378-8ce1-ee0dd6057665 admin admin] PATH_INFO: `/v3/users` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:06.539065 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443fcac8-5b52-4378-8ce1-ee0dd6057665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:06.540993 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443fcac8-5b52-4378-8ce1-ee0dd6057665 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:06.605642 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1393/2804] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:26:06 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:26:06.609744 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf206d3c-e1ca-4698-aaca-b021cd53b554 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:06.620981 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf206d3c-e1ca-4698-aaca-b021cd53b554 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:06.625612 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054de36-dee6-486c-9383-95f49db609c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:06.626450 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054de36-dee6-486c-9383-95f49db609c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:06.626743 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054de36-dee6-486c-9383-95f49db609c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:06.627026 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054de36-dee6-486c-9383-95f49db609c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:06.627809 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054de36-dee6-486c-9383-95f49db609c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:06.629441 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054de36-dee6-486c-9383-95f49db609c2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:06.640711 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1412/2805] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:26:06.644619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443fcac8-5b52-4378-8ce1-ee0dd6057665 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:06.655055 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443fcac8-5b52-4378-8ce1-ee0dd6057665 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:06.659818 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be69f0-c12a-4b4b-8ba3-bde7327546f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:06.660413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be69f0-c12a-4b4b-8ba3-bde7327546f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:06.660703 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be69f0-c12a-4b4b-8ba3-bde7327546f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:06.660983 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be69f0-c12a-4b4b-8ba3-bde7327546f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:06.661868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be69f0-c12a-4b4b-8ba3-bde7327546f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:06.663511 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be69f0-c12a-4b4b-8ba3-bde7327546f1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:06.673395 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1394/2806] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:26:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:26:06.678358 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054de36-dee6-486c-9383-95f49db609c2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:06.688596 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054de36-dee6-486c-9383-95f49db609c2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:06.694060 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888116cc-1d97-4fd5-bce6-e897874b63f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:06.694977 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888116cc-1d97-4fd5-bce6-e897874b63f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:06.695264 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888116cc-1d97-4fd5-bce6-e897874b63f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:06.695545 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888116cc-1d97-4fd5-bce6-e897874b63f4 admin admin] PATH_INFO: `/v3/projects/05169e1eba7b4ef89431e188f204b76f/users/8cc20cdc574f43909d6ae0c9ec23d4dd/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:06.711783 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888116cc-1d97-4fd5-bce6-e897874b63f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05169e1eba7b4ef89431e188f204b76f, user_id=8cc20cdc574f43909d6ae0c9ec23d4dd, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:06.716293 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888116cc-1d97-4fd5-bce6-e897874b63f4 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:06.822673 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1413/2807] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:26:06 2022] PUT /identity/v3/projects/05169e1eba7b4ef89431e188f204b76f/users/8cc20cdc574f43909d6ae0c9ec23d4dd/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:26:06.833098 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80210bce-e122-40a0-9eec-f0dddc5c89b4 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:06.833450 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80210bce-e122-40a0-9eec-f0dddc5c89b4 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:06.833996 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80210bce-e122-40a0-9eec-f0dddc5c89b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:06.861434 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-80210bce-e122-40a0-9eec-f0dddc5c89b4 None None] MFA Rules not processed for user `8cc20cdc574f43909d6ae0c9ec23d4dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:06.862669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-80210bce-e122-40a0-9eec-f0dddc5c89b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:06.930355 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1395/2808] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:26:06.941872 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888116cc-1d97-4fd5-bce6-e897874b63f4 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:06.943782 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-888116cc-1d97-4fd5-bce6-e897874b63f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:06.965497 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:07.001052 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888116cc-1d97-4fd5-bce6-e897874b63f4 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:07.004748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a386f6-6fdf-4c82-8eb3-491ed957c98f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:07.005458 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a386f6-6fdf-4c82-8eb3-491ed957c98f service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:07.005746 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a386f6-6fdf-4c82-8eb3-491ed957c98f service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:07.006025 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a386f6-6fdf-4c82-8eb3-491ed957c98f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:07.014327 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a386f6-6fdf-4c82-8eb3-491ed957c98f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:07.015793 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a386f6-6fdf-4c82-8eb3-491ed957c98f service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:07.024831 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1414/2809] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:26:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:26:10.925347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80210bce-e122-40a0-9eec-f0dddc5c89b4 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:10.945608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:10.992266 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80210bce-e122-40a0-9eec-f0dddc5c89b4 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:10.996983 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e2ffe-45f6-47b5-9182-bfb3737c1f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:10.997886 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e2ffe-45f6-47b5-9182-bfb3737c1f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:10.998169 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e2ffe-45f6-47b5-9182-bfb3737c1f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:10.998483 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e2ffe-45f6-47b5-9182-bfb3737c1f52 admin admin] PATH_INFO: `/v3/users/8cc20cdc574f43909d6ae0c9ec23d4dd` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:10.999685 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e2ffe-45f6-47b5-9182-bfb3737c1f52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cc20cdc574f43909d6ae0c9ec23d4dd)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:11.001611 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e2ffe-45f6-47b5-9182-bfb3737c1f52 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:11.121277 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a386f6-6fdf-4c82-8eb3-491ed957c98f service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:11.122797 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-70a386f6-6fdf-4c82-8eb3-491ed957c98f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:11.139440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a386f6-6fdf-4c82-8eb3-491ed957c98f service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:11.144245 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45884b-424e-4c6d-849e-4938b61247c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:11.145121 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45884b-424e-4c6d-849e-4938b61247c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:11.145408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45884b-424e-4c6d-849e-4938b61247c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:11.145746 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45884b-424e-4c6d-849e-4938b61247c8 admin admin] PATH_INFO: `/v3/users/8ec358de5080410bb5d578bd59df67db` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:11.148779 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45884b-424e-4c6d-849e-4938b61247c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ec358de5080410bb5d578bd59df67db)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:11.149146 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45884b-424e-4c6d-849e-4938b61247c8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:11.185580 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-af1e2ffe-45f6-47b5-9182-bfb3737c1f52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cc20cdc574f43909d6ae0c9ec23d4dd'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:11.198282 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-af1e2ffe-45f6-47b5-9182-bfb3737c1f52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cc20cdc574f43909d6ae0c9ec23d4dd'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:11.266849 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-af1e2ffe-45f6-47b5-9182-bfb3737c1f52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cc20cdc574f43909d6ae0c9ec23d4dd'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:11.267487 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-af1e2ffe-45f6-47b5-9182-bfb3737c1f52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cc20cdc574f43909d6ae0c9ec23d4dd'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:11.268090 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-af1e2ffe-45f6-47b5-9182-bfb3737c1f52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cc20cdc574f43909d6ae0c9ec23d4dd'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:11.281053 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1396/2810] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:26:10 2022] DELETE /identity/v3/users/8cc20cdc574f43909d6ae0c9ec23d4dd => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:26:11.285323 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a5a30-c90c-4d61-9c62-bd059323c69c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:11.285614 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a5a30-c90c-4d61-9c62-bd059323c69c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:11.285900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a5a30-c90c-4d61-9c62-bd059323c69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:11.302860 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-733a5a30-c90c-4d61-9c62-bd059323c69c None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:11.303822 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-733a5a30-c90c-4d61-9c62-bd059323c69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:11.337669 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-bb45884b-424e-4c6d-849e-4938b61247c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ec358de5080410bb5d578bd59df67db'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:11.350588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-bb45884b-424e-4c6d-849e-4938b61247c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ec358de5080410bb5d578bd59df67db'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:11.351316 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-bb45884b-424e-4c6d-849e-4938b61247c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ec358de5080410bb5d578bd59df67db'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:11.351932 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-bb45884b-424e-4c6d-849e-4938b61247c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ec358de5080410bb5d578bd59df67db'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:11.358817 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-bb45884b-424e-4c6d-849e-4938b61247c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ec358de5080410bb5d578bd59df67db'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:11.366297 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1397/2811] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:26:11.376264 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a5a30-c90c-4d61-9c62-bd059323c69c None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:11.377593 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-733a5a30-c90c-4d61-9c62-bd059323c69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:11.419347 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:11.434691 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a5a30-c90c-4d61-9c62-bd059323c69c None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:11.438425 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b11fe6-8f91-4475-a605-836558a401ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:11.439024 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b11fe6-8f91-4475-a605-836558a401ee service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:11.439312 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b11fe6-8f91-4475-a605-836558a401ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:11.439592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b11fe6-8f91-4475-a605-836558a401ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:11.448642 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b11fe6-8f91-4475-a605-836558a401ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:11.450075 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b11fe6-8f91-4475-a605-836558a401ee service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:11.459770 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1398/2812] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:26:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:26:11.490556 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1415/2813] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:26:11 2022] DELETE /identity/v3/users/8ec358de5080410bb5d578bd59df67db => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:26:11.740014 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b11fe6-8f91-4475-a605-836558a401ee service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:11.741674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-01b11fe6-8f91-4475-a605-836558a401ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:11.755831 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b11fe6-8f91-4475-a605-836558a401ee service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:11.834607 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb09a4-f76c-4854-bda2-1cde5aecd5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:11.835479 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb09a4-f76c-4854-bda2-1cde5aecd5b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:11.835762 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb09a4-f76c-4854-bda2-1cde5aecd5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:11.836040 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb09a4-f76c-4854-bda2-1cde5aecd5b3 admin admin] PATH_INFO: `/v3/projects/b3bcb19357c24ccba28c50654673d103` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:11.837239 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb09a4-f76c-4854-bda2-1cde5aecd5b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3bcb19357c24ccba28c50654673d103)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:11.839191 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb09a4-f76c-4854-bda2-1cde5aecd5b3 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:11.853891 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45884b-424e-4c6d-849e-4938b61247c8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:11.855584 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-bb45884b-424e-4c6d-849e-4938b61247c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:11.870796 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45884b-424e-4c6d-849e-4938b61247c8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:11.875700 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c6429-0672-4144-b220-56741b6a05f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:11.876810 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c6429-0672-4144-b220-56741b6a05f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:11.877171 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c6429-0672-4144-b220-56741b6a05f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:11.877527 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c6429-0672-4144-b220-56741b6a05f5 admin admin] PATH_INFO: `/v3/projects/05169e1eba7b4ef89431e188f204b76f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:11.878995 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c6429-0672-4144-b220-56741b6a05f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05169e1eba7b4ef89431e188f204b76f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:11.881421 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957c6429-0672-4144-b220-56741b6a05f5 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:11.960328 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3ebb09a4-f76c-4854-bda2-1cde5aecd5b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3bcb19357c24ccba28c50654673d103', 'request_id': 'req-3ebb09a4-f76c-4854-bda2-1cde5aecd5b3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:11.983455 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-957c6429-0672-4144-b220-56741b6a05f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05169e1eba7b4ef89431e188f204b76f', 'request_id': 'req-957c6429-0672-4144-b220-56741b6a05f5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:11.989708 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-957c6429-0672-4144-b220-56741b6a05f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05169e1eba7b4ef89431e188f204b76f', 'request_id': 'req-957c6429-0672-4144-b220-56741b6a05f5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:11.995423 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-957c6429-0672-4144-b220-56741b6a05f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05169e1eba7b4ef89431e188f204b76f', 'request_id': 'req-957c6429-0672-4144-b220-56741b6a05f5'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:12.045366 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3ebb09a4-f76c-4854-bda2-1cde5aecd5b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3bcb19357c24ccba28c50654673d103', 'request_id': 'req-3ebb09a4-f76c-4854-bda2-1cde5aecd5b3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:12.051745 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3ebb09a4-f76c-4854-bda2-1cde5aecd5b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3bcb19357c24ccba28c50654673d103', 'request_id': 'req-3ebb09a4-f76c-4854-bda2-1cde5aecd5b3'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:12.063608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3ebb09a4-f76c-4854-bda2-1cde5aecd5b3 admin admin] The token cache is being invalidate because project b3bcb19357c24ccba28c50654673d103 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:26:12.063900 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3ebb09a4-f76c-4854-bda2-1cde5aecd5b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:12.064561 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-3ebb09a4-f76c-4854-bda2-1cde5aecd5b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:12.066132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1399/2814] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:26:11 2022] DELETE /identity/v3/projects/b3bcb19357c24ccba28c50654673d103 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:26:12.070880 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb09a4-f76c-4854-bda2-1cde5aecd5b3 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:12.072150 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3ebb09a4-f76c-4854-bda2-1cde5aecd5b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:12.136993 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-957c6429-0672-4144-b220-56741b6a05f5 admin admin] The token cache is being invalidate because project 05169e1eba7b4ef89431e188f204b76f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:26:12.137602 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-957c6429-0672-4144-b220-56741b6a05f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:12.138095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-957c6429-0672-4144-b220-56741b6a05f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:12.138522 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb09a4-f76c-4854-bda2-1cde5aecd5b3 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:12.139791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1416/2815] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:26:11 2022] DELETE /identity/v3/projects/05169e1eba7b4ef89431e188f204b76f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:26:12.144525 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9e197-d198-4df8-8bb2-629eba01da2f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:12.148224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9e197-d198-4df8-8bb2-629eba01da2f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:12.148224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9e197-d198-4df8-8bb2-629eba01da2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:12.152853 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:12.165535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-2bb9e197-d198-4df8-8bb2-629eba01da2f None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:12.166846 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2bb9e197-d198-4df8-8bb2-629eba01da2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:12.179954 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:12.197400 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e273cb-4f6c-4371-8ca0-eba170957d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:12.198337 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e273cb-4f6c-4371-8ca0-eba170957d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:12.198600 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e273cb-4f6c-4371-8ca0-eba170957d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:12.198883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e273cb-4f6c-4371-8ca0-eba170957d7e admin admin] PATH_INFO: `/v3/users/3505f9823bf4469293523eee1d4638ff` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:12.200120 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e273cb-4f6c-4371-8ca0-eba170957d7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3505f9823bf4469293523eee1d4638ff)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:12.202075 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e273cb-4f6c-4371-8ca0-eba170957d7e admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:12.228645 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1417/2816] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:26:12.237158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981610f5-491a-4ed2-ac38-66e89ceff697 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:12.237450 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981610f5-491a-4ed2-ac38-66e89ceff697 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:12.237797 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981610f5-491a-4ed2-ac38-66e89ceff697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:12.257560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-981610f5-491a-4ed2-ac38-66e89ceff697 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:12.258803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-981610f5-491a-4ed2-ac38-66e89ceff697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:12.267514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1418/2817] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:26:12.272209 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981610f5-491a-4ed2-ac38-66e89ceff697 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:12.281240 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981610f5-491a-4ed2-ac38-66e89ceff697 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:12.285945 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30de6e-6139-4251-a286-e6eb3ffca724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:12.286745 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30de6e-6139-4251-a286-e6eb3ffca724 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:12.287186 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30de6e-6139-4251-a286-e6eb3ffca724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:12.287186 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b30de6e-6139-4251-a286-e6eb3ffca724 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:12.288003 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30de6e-6139-4251-a286-e6eb3ffca724 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:26:12.288314 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30de6e-6139-4251-a286-e6eb3ffca724 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:12.289929 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b30de6e-6139-4251-a286-e6eb3ffca724 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:12.301238 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1419/2818] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 04:26:12.305860 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30de6e-6139-4251-a286-e6eb3ffca724 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:12.315356 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b30de6e-6139-4251-a286-e6eb3ffca724 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:12.320560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446165b-7076-4a86-ab5c-cd7a9863f4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:12.321190 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446165b-7076-4a86-ab5c-cd7a9863f4d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:12.321477 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446165b-7076-4a86-ab5c-cd7a9863f4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:12.321752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9446165b-7076-4a86-ab5c-cd7a9863f4d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:12.322466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446165b-7076-4a86-ab5c-cd7a9863f4d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:12.324367 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9446165b-7076-4a86-ab5c-cd7a9863f4d3 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:12.334584 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-27e273cb-4f6c-4371-8ca0-eba170957d7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3505f9823bf4469293523eee1d4638ff'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:12.346751 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-27e273cb-4f6c-4371-8ca0-eba170957d7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3505f9823bf4469293523eee1d4638ff'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:12.406240 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1420/2819] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:26:12 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:26:12.409730 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446165b-7076-4a86-ab5c-cd7a9863f4d3 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:12.422433 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-27e273cb-4f6c-4371-8ca0-eba170957d7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3505f9823bf4469293523eee1d4638ff'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:12.423493 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-27e273cb-4f6c-4371-8ca0-eba170957d7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3505f9823bf4469293523eee1d4638ff'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:12.424203 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-27e273cb-4f6c-4371-8ca0-eba170957d7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3505f9823bf4469293523eee1d4638ff'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:12.436817 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1400/2820] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:26:12 2022] DELETE /identity/v3/users/3505f9823bf4469293523eee1d4638ff => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:26:12.476734 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9446165b-7076-4a86-ab5c-cd7a9863f4d3 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:12.534029 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c586e-459f-4339-b04e-fc547d4eae7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:12.534726 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c586e-459f-4339-b04e-fc547d4eae7e admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:12.535016 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c586e-459f-4339-b04e-fc547d4eae7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:12.535296 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c586e-459f-4339-b04e-fc547d4eae7e admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:12.535992 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c586e-459f-4339-b04e-fc547d4eae7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:12.538024 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c586e-459f-4339-b04e-fc547d4eae7e admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:12.617236 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1421/2821] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:26:12 2022] POST /identity/v3/users => generated 283 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:26:12.621502 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e273cb-4f6c-4371-8ca0-eba170957d7e admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:12.622951 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-27e273cb-4f6c-4371-8ca0-eba170957d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:12.636118 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e273cb-4f6c-4371-8ca0-eba170957d7e admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:12.640717 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd238ce4-18a1-4bd9-ab73-ca119e148c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:12.641408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd238ce4-18a1-4bd9-ab73-ca119e148c60 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:12.641686 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd238ce4-18a1-4bd9-ab73-ca119e148c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:12.641956 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd238ce4-18a1-4bd9-ab73-ca119e148c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:12.642715 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd238ce4-18a1-4bd9-ab73-ca119e148c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:12.644418 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd238ce4-18a1-4bd9-ab73-ca119e148c60 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:12.658674 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1401/2822] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:26:12.660058 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c586e-459f-4339-b04e-fc547d4eae7e admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:12.666629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd238ce4-18a1-4bd9-ab73-ca119e148c60 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:12.669373 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-cd238ce4-18a1-4bd9-ab73-ca119e148c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:12.678492 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c586e-459f-4339-b04e-fc547d4eae7e admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:12.686231 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce8a85-f859-4e71-b531-598cf0195183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:12.688217 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce8a85-f859-4e71-b531-598cf0195183 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:12.688749 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce8a85-f859-4e71-b531-598cf0195183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:12.689258 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ce8a85-f859-4e71-b531-598cf0195183 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:12.690678 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce8a85-f859-4e71-b531-598cf0195183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:12.692698 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd238ce4-18a1-4bd9-ab73-ca119e148c60 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:12.693791 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ce8a85-f859-4e71-b531-598cf0195183 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:12.697766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17c7e6-c9b5-4a1b-b7be-5640ae69b97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:12.698568 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17c7e6-c9b5-4a1b-b7be-5640ae69b97c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:12.699010 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17c7e6-c9b5-4a1b-b7be-5640ae69b97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:12.699251 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17c7e6-c9b5-4a1b-b7be-5640ae69b97c admin admin] PATH_INFO: `/v3/projects/b61d9a36f41e4f7d9967cc973e0864d9` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:12.700536 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17c7e6-c9b5-4a1b-b7be-5640ae69b97c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b61d9a36f41e4f7d9967cc973e0864d9)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:12.702413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17c7e6-c9b5-4a1b-b7be-5640ae69b97c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:12.709908 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1422/2823] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 04:26:12.714635 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce8a85-f859-4e71-b531-598cf0195183 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:12.720433 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:12.726451 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ce8a85-f859-4e71-b531-598cf0195183 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:12.731953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8ce05-25ce-401f-91eb-8a498d1f1263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:12.736091 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8ce05-25ce-401f-91eb-8a498d1f1263 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:12.736091 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8ce05-25ce-401f-91eb-8a498d1f1263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:12.736091 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8ce05-25ce-401f-91eb-8a498d1f1263 admin admin] PATH_INFO: `/v3/projects/ffaa6d4e4e66434792fbb65b9a2a2075/users/2b07b50c71b2412183f8683730506e03/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:12.749055 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8ce05-25ce-401f-91eb-8a498d1f1263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffaa6d4e4e66434792fbb65b9a2a2075, user_id=2b07b50c71b2412183f8683730506e03, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:12.751771 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8ce05-25ce-401f-91eb-8a498d1f1263 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:12.851515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1423/2824] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:26:12 2022] PUT /identity/v3/projects/ffaa6d4e4e66434792fbb65b9a2a2075/users/2b07b50c71b2412183f8683730506e03/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:26:12.859616 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef17c7e6-c9b5-4a1b-b7be-5640ae69b97c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b61d9a36f41e4f7d9967cc973e0864d9', 'request_id': 'req-ef17c7e6-c9b5-4a1b-b7be-5640ae69b97c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:12.861746 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703adbfb-ea94-45b1-9839-cde433099fee None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:12.862035 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703adbfb-ea94-45b1-9839-cde433099fee None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:12.862321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703adbfb-ea94-45b1-9839-cde433099fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:12.888417 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-703adbfb-ea94-45b1-9839-cde433099fee None None] MFA Rules not processed for user `2b07b50c71b2412183f8683730506e03`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:12.889619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-703adbfb-ea94-45b1-9839-cde433099fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:12.929628 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef17c7e6-c9b5-4a1b-b7be-5640ae69b97c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b61d9a36f41e4f7d9967cc973e0864d9', 'request_id': 'req-ef17c7e6-c9b5-4a1b-b7be-5640ae69b97c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:12.935999 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef17c7e6-c9b5-4a1b-b7be-5640ae69b97c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b61d9a36f41e4f7d9967cc973e0864d9', 'request_id': 'req-ef17c7e6-c9b5-4a1b-b7be-5640ae69b97c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:12.949533 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef17c7e6-c9b5-4a1b-b7be-5640ae69b97c admin admin] The token cache is being invalidate because project b61d9a36f41e4f7d9967cc973e0864d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:26:12.949817 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef17c7e6-c9b5-4a1b-b7be-5640ae69b97c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:12.950553 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ef17c7e6-c9b5-4a1b-b7be-5640ae69b97c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:12.952194 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1402/2825] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:26:12 2022] DELETE /identity/v3/projects/b61d9a36f41e4f7d9967cc973e0864d9 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:26:12.955485 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1424/2826] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:26:12.957967 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0d1eb-d498-4c6c-b448-4edb074d0a83 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:12.959430 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0d1eb-d498-4c6c-b448-4edb074d0a83 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:12.959430 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0d1eb-d498-4c6c-b448-4edb074d0a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:12.967270 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703adbfb-ea94-45b1-9839-cde433099fee None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:12.968760 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-703adbfb-ea94-45b1-9839-cde433099fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:12.977152 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-9be0d1eb-d498-4c6c-b448-4edb074d0a83 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:12.978346 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-9be0d1eb-d498-4c6c-b448-4edb074d0a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:12.982910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:13.028013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703adbfb-ea94-45b1-9839-cde433099fee None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:13.032010 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa61e24-074a-45ec-8fb4-791514ae2f39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:13.032706 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa61e24-074a-45ec-8fb4-791514ae2f39 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:13.033046 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa61e24-074a-45ec-8fb4-791514ae2f39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:13.033335 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa61e24-074a-45ec-8fb4-791514ae2f39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:13.035347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-9aa61e24-074a-45ec-8fb4-791514ae2f39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:13.042718 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1403/2827] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:26:13.047619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa61e24-074a-45ec-8fb4-791514ae2f39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:13.048402 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a52ba3-0668-4a32-b3ad-ed95e814c60b None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:13.048694 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a52ba3-0668-4a32-b3ad-ed95e814c60b None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:13.048977 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a52ba3-0668-4a32-b3ad-ed95e814c60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:13.049172 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa61e24-074a-45ec-8fb4-791514ae2f39 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:13.058686 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1425/2828] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:26:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:26:13.068066 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-99a52ba3-0668-4a32-b3ad-ed95e814c60b None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:13.069378 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-99a52ba3-0668-4a32-b3ad-ed95e814c60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:13.078394 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1404/2829] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:26:13.082933 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa61e24-074a-45ec-8fb4-791514ae2f39 service cinder] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:13.094504 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa61e24-074a-45ec-8fb4-791514ae2f39 service cinder] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:13.099202 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b16864-d95c-450f-8427-5b57d9bcb4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:13.099956 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b16864-d95c-450f-8427-5b57d9bcb4df admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:13.100195 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b16864-d95c-450f-8427-5b57d9bcb4df admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:13.100477 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b16864-d95c-450f-8427-5b57d9bcb4df admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:13.101238 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b16864-d95c-450f-8427-5b57d9bcb4df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:26:13.101552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b16864-d95c-450f-8427-5b57d9bcb4df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:13.103225 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b16864-d95c-450f-8427-5b57d9bcb4df admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:13.119141 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1426/2830] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:26:13.120232 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a52ba3-0668-4a32-b3ad-ed95e814c60b None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:13.129355 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a52ba3-0668-4a32-b3ad-ed95e814c60b None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:13.134913 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93683fc4-f03d-42c6-bc02-9cb018e1072a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:13.135565 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93683fc4-f03d-42c6-bc02-9cb018e1072a admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:13.135929 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93683fc4-f03d-42c6-bc02-9cb018e1072a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:13.136125 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93683fc4-f03d-42c6-bc02-9cb018e1072a admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:13.136964 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93683fc4-f03d-42c6-bc02-9cb018e1072a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:13.138945 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93683fc4-f03d-42c6-bc02-9cb018e1072a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:13.222433 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1405/2831] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:26:13 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:26:13.227502 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b16864-d95c-450f-8427-5b57d9bcb4df admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:13.237631 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b16864-d95c-450f-8427-5b57d9bcb4df admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:13.291176 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:13.296994 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c9794-230d-46ac-a9ac-c42228e3da53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:13.297667 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c9794-230d-46ac-a9ac-c42228e3da53 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:13.297956 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c9794-230d-46ac-a9ac-c42228e3da53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:13.298243 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c9794-230d-46ac-a9ac-c42228e3da53 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:13.298983 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c9794-230d-46ac-a9ac-c42228e3da53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:13.300985 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c9794-230d-46ac-a9ac-c42228e3da53 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:13.401171 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1427/2832] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:26:13 2022] POST /identity/v3/users => generated 292 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:26:13.405288 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93683fc4-f03d-42c6-bc02-9cb018e1072a admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:13.416503 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93683fc4-f03d-42c6-bc02-9cb018e1072a admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:13.421215 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e13f78-d74f-48a2-a71b-09776696b458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:13.421869 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e13f78-d74f-48a2-a71b-09776696b458 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:13.425869 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e13f78-d74f-48a2-a71b-09776696b458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:13.425869 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e13f78-d74f-48a2-a71b-09776696b458 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:13.425869 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e13f78-d74f-48a2-a71b-09776696b458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:13.425869 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e13f78-d74f-48a2-a71b-09776696b458 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:13.435706 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1406/2833] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:26:13.439736 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c9794-230d-46ac-a9ac-c42228e3da53 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:13.449874 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c9794-230d-46ac-a9ac-c42228e3da53 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:13.454713 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ad568-34e3-4251-b8a2-3d4372aca733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:13.455464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ad568-34e3-4251-b8a2-3d4372aca733 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:13.455747 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ad568-34e3-4251-b8a2-3d4372aca733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:13.456021 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ad568-34e3-4251-b8a2-3d4372aca733 admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:13.456752 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ad568-34e3-4251-b8a2-3d4372aca733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:13.458924 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ad568-34e3-4251-b8a2-3d4372aca733 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:13.469857 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1428/2834] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:26:13.474494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e13f78-d74f-48a2-a71b-09776696b458 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:13.484437 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e13f78-d74f-48a2-a71b-09776696b458 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:13.489087 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016604c3-936c-4a0b-8eb5-2f165125c829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:13.489999 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016604c3-936c-4a0b-8eb5-2f165125c829 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:13.490281 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016604c3-936c-4a0b-8eb5-2f165125c829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:13.490590 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016604c3-936c-4a0b-8eb5-2f165125c829 admin admin] PATH_INFO: `/v3/projects/0453eae082704e9cbdb3d1efa1aa28c8/users/52e74182692a415581b4d8797fe10e5a/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:13.504793 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016604c3-936c-4a0b-8eb5-2f165125c829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0453eae082704e9cbdb3d1efa1aa28c8, user_id=52e74182692a415581b4d8797fe10e5a, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:13.507636 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016604c3-936c-4a0b-8eb5-2f165125c829 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:13.605270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1407/2835] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:26:13 2022] PUT /identity/v3/projects/0453eae082704e9cbdb3d1efa1aa28c8/users/52e74182692a415581b4d8797fe10e5a/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:26:13.616685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534fc64-d304-4242-8ba9-ca6db457927f None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:13.617009 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534fc64-d304-4242-8ba9-ca6db457927f None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:13.617287 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534fc64-d304-4242-8ba9-ca6db457927f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:13.643902 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-5534fc64-d304-4242-8ba9-ca6db457927f None None] MFA Rules not processed for user `52e74182692a415581b4d8797fe10e5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:13.645114 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5534fc64-d304-4242-8ba9-ca6db457927f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:13.717168 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1429/2836] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:26:13.728999 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016604c3-936c-4a0b-8eb5-2f165125c829 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:13.730807 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-016604c3-936c-4a0b-8eb5-2f165125c829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:13.791415 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016604c3-936c-4a0b-8eb5-2f165125c829 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:13.795273 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb057769-6d03-45df-a0aa-daa0615638ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:13.795926 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb057769-6d03-45df-a0aa-daa0615638ac service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:13.796217 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb057769-6d03-45df-a0aa-daa0615638ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:13.796500 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb057769-6d03-45df-a0aa-daa0615638ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:13.805208 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb057769-6d03-45df-a0aa-daa0615638ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:13.806687 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb057769-6d03-45df-a0aa-daa0615638ac service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:13.815766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1408/2837] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:26:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:26:23.523821 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534fc64-d304-4242-8ba9-ca6db457927f None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:23.589926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534fc64-d304-4242-8ba9-ca6db457927f None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:23.598684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64ccad-b25d-4e8e-b1d4-3a0d7ad5d56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:23.598684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64ccad-b25d-4e8e-b1d4-3a0d7ad5d56f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:23.598684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64ccad-b25d-4e8e-b1d4-3a0d7ad5d56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:23.598684 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64ccad-b25d-4e8e-b1d4-3a0d7ad5d56f admin admin] PATH_INFO: `/v3/users/52e74182692a415581b4d8797fe10e5a` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:23.599154 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64ccad-b25d-4e8e-b1d4-3a0d7ad5d56f admin admin] RBAC: Authorizing `identity:delete_user(user_id=52e74182692a415581b4d8797fe10e5a)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:23.600027 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64ccad-b25d-4e8e-b1d4-3a0d7ad5d56f admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:23.762944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7d64ccad-b25d-4e8e-b1d4-3a0d7ad5d56f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52e74182692a415581b4d8797fe10e5a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:23.776652 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7d64ccad-b25d-4e8e-b1d4-3a0d7ad5d56f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52e74182692a415581b4d8797fe10e5a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:23.777507 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7d64ccad-b25d-4e8e-b1d4-3a0d7ad5d56f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52e74182692a415581b4d8797fe10e5a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:23.778156 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7d64ccad-b25d-4e8e-b1d4-3a0d7ad5d56f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52e74182692a415581b4d8797fe10e5a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:23.780288 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:23.785601 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-7d64ccad-b25d-4e8e-b1d4-3a0d7ad5d56f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52e74182692a415581b4d8797fe10e5a'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:23.811610 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=76,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:23.891207 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1430/2838] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:26:23 2022] DELETE /identity/v3/users/52e74182692a415581b4d8797fe10e5a => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:26:23.895922 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d00f58-e3e4-4c2c-aa8a-9fa760a14ab3 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:23.896481 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d00f58-e3e4-4c2c-aa8a-9fa760a14ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:23.896897 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d00f58-e3e4-4c2c-aa8a-9fa760a14ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:23.914936 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-a4d00f58-e3e4-4c2c-aa8a-9fa760a14ab3 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:23.916157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a4d00f58-e3e4-4c2c-aa8a-9fa760a14ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:23.980506 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1409/2839] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:26:23.990150 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64ccad-b25d-4e8e-b1d4-3a0d7ad5d56f admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:23.991836 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-7d64ccad-b25d-4e8e-b1d4-3a0d7ad5d56f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:24.050136 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64ccad-b25d-4e8e-b1d4-3a0d7ad5d56f admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:24.054158 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8f1995-708e-40b8-a5ab-4dba6c316fda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:24.054840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f1995-708e-40b8-a5ab-4dba6c316fda service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:24.062440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f1995-708e-40b8-a5ab-4dba6c316fda service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:24.062440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8f1995-708e-40b8-a5ab-4dba6c316fda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:24.064789 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f1995-708e-40b8-a5ab-4dba6c316fda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:24.066268 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8f1995-708e-40b8-a5ab-4dba6c316fda service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:24.076352 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1431/2840] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:26:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:26:24.503394 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d00f58-e3e4-4c2c-aa8a-9fa760a14ab3 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:24.504805 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a4d00f58-e3e4-4c2c-aa8a-9fa760a14ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:24.517278 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d00f58-e3e4-4c2c-aa8a-9fa760a14ab3 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:24.523058 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0deb51-26aa-47a6-9eaa-0077985a9e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:24.524168 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0deb51-26aa-47a6-9eaa-0077985a9e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:24.524473 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0deb51-26aa-47a6-9eaa-0077985a9e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:24.524730 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0deb51-26aa-47a6-9eaa-0077985a9e1c admin admin] PATH_INFO: `/v3/projects/0453eae082704e9cbdb3d1efa1aa28c8` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:24.526183 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0deb51-26aa-47a6-9eaa-0077985a9e1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0453eae082704e9cbdb3d1efa1aa28c8)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:24.528130 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0deb51-26aa-47a6-9eaa-0077985a9e1c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:24.622198 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fb0deb51-26aa-47a6-9eaa-0077985a9e1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0453eae082704e9cbdb3d1efa1aa28c8', 'request_id': 'req-fb0deb51-26aa-47a6-9eaa-0077985a9e1c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:24.661766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fb0deb51-26aa-47a6-9eaa-0077985a9e1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0453eae082704e9cbdb3d1efa1aa28c8', 'request_id': 'req-fb0deb51-26aa-47a6-9eaa-0077985a9e1c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:24.668442 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fb0deb51-26aa-47a6-9eaa-0077985a9e1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0453eae082704e9cbdb3d1efa1aa28c8', 'request_id': 'req-fb0deb51-26aa-47a6-9eaa-0077985a9e1c'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:24.680224 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fb0deb51-26aa-47a6-9eaa-0077985a9e1c admin admin] The token cache is being invalidate because project 0453eae082704e9cbdb3d1efa1aa28c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:26:24.680510 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fb0deb51-26aa-47a6-9eaa-0077985a9e1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:24.681136 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-fb0deb51-26aa-47a6-9eaa-0077985a9e1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:24.682701 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1410/2841] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:26:24 2022] DELETE /identity/v3/projects/0453eae082704e9cbdb3d1efa1aa28c8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:26:24.687608 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a8677-b984-4a44-940c-bfa787e14af6 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:24.688070 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a8677-b984-4a44-940c-bfa787e14af6 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:24.688185 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249a8677-b984-4a44-940c-bfa787e14af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:24.707006 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-249a8677-b984-4a44-940c-bfa787e14af6 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:24.708199 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-249a8677-b984-4a44-940c-bfa787e14af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:24.771459 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1432/2842] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:26:24.779194 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b457b-4121-44e6-bc10-e872bd65e714 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:24.779487 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b457b-4121-44e6-bc10-e872bd65e714 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:24.779766 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b457b-4121-44e6-bc10-e872bd65e714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:24.799157 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-675b457b-4121-44e6-bc10-e872bd65e714 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:24.800359 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-675b457b-4121-44e6-bc10-e872bd65e714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:24.809361 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1411/2843] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:26:24.813705 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a8677-b984-4a44-940c-bfa787e14af6 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:24.823670 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249a8677-b984-4a44-940c-bfa787e14af6 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:24.828237 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ddcc91-b34a-45f6-a0dc-d36c1bae211b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:24.828889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ddcc91-b34a-45f6-a0dc-d36c1bae211b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:24.829178 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ddcc91-b34a-45f6-a0dc-d36c1bae211b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:24.829460 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ddcc91-b34a-45f6-a0dc-d36c1bae211b admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:24.830187 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ddcc91-b34a-45f6-a0dc-d36c1bae211b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:26:24.830533 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ddcc91-b34a-45f6-a0dc-d36c1bae211b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:24.832141 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ddcc91-b34a-45f6-a0dc-d36c1bae211b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:24.843560 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1433/2844] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:26:24.850939 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b457b-4121-44e6-bc10-e872bd65e714 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:24.860071 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b457b-4121-44e6-bc10-e872bd65e714 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:24.864652 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ed916-50a4-4a80-82ee-cf4147698d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:24.866391 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ed916-50a4-4a80-82ee-cf4147698d35 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:24.866685 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ed916-50a4-4a80-82ee-cf4147698d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:24.866986 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ed916-50a4-4a80-82ee-cf4147698d35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:24.867677 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ed916-50a4-4a80-82ee-cf4147698d35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:24.869572 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ed916-50a4-4a80-82ee-cf4147698d35 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:24.949092 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1412/2845] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:26:24 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:26:24.953263 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ddcc91-b34a-45f6-a0dc-d36c1bae211b admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:24.963469 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ddcc91-b34a-45f6-a0dc-d36c1bae211b admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:25.019233 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1a128-a774-47a9-a390-570a8c18bf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:25.019825 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1a128-a774-47a9-a390-570a8c18bf3a admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:25.020149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1a128-a774-47a9-a390-570a8c18bf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:25.020435 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d1a128-a774-47a9-a390-570a8c18bf3a admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:25.021169 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1a128-a774-47a9-a390-570a8c18bf3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:25.023255 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d1a128-a774-47a9-a390-570a8c18bf3a admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:25.118655 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1434/2846] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:26:24 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:26:25.122772 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ed916-50a4-4a80-82ee-cf4147698d35 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:25.134073 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ed916-50a4-4a80-82ee-cf4147698d35 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:25.138652 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040ad28-ec08-4ba5-9a32-54e45ad8318c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:25.139304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040ad28-ec08-4ba5-9a32-54e45ad8318c admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:25.139664 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040ad28-ec08-4ba5-9a32-54e45ad8318c admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:25.139942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040ad28-ec08-4ba5-9a32-54e45ad8318c admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:25.140671 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040ad28-ec08-4ba5-9a32-54e45ad8318c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:25.142264 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040ad28-ec08-4ba5-9a32-54e45ad8318c admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:25.154715 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1413/2847] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:26:25.158713 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1a128-a774-47a9-a390-570a8c18bf3a admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:25.165311 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:25.169040 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d1a128-a774-47a9-a390-570a8c18bf3a admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:25.173618 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c234ab-7fcb-4800-b747-e5dd327d417b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:25.174299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c234ab-7fcb-4800-b747-e5dd327d417b admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:25.174613 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c234ab-7fcb-4800-b747-e5dd327d417b admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:25.174893 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c234ab-7fcb-4800-b747-e5dd327d417b admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:25.175610 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c234ab-7fcb-4800-b747-e5dd327d417b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:25.177201 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c234ab-7fcb-4800-b747-e5dd327d417b admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:25.188422 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1435/2848] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:26:25.203819 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040ad28-ec08-4ba5-9a32-54e45ad8318c admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:25.213844 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040ad28-ec08-4ba5-9a32-54e45ad8318c admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:25.218361 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a864bf-a637-4f1d-b06d-d4349c9c5009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:25.219300 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a864bf-a637-4f1d-b06d-d4349c9c5009 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:25.219579 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a864bf-a637-4f1d-b06d-d4349c9c5009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:25.219852 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a864bf-a637-4f1d-b06d-d4349c9c5009 admin admin] PATH_INFO: `/v3/projects/c07ef3d97ff54852b43087b7d223a6a0/users/d0aa07f92f994c1c933b9ff31623319d/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:25.234570 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a864bf-a637-4f1d-b06d-d4349c9c5009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c07ef3d97ff54852b43087b7d223a6a0, user_id=d0aa07f92f994c1c933b9ff31623319d, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:25.237286 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a864bf-a637-4f1d-b06d-d4349c9c5009 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:25.337072 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1414/2849] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:26:25 2022] PUT /identity/v3/projects/c07ef3d97ff54852b43087b7d223a6a0/users/d0aa07f92f994c1c933b9ff31623319d/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:26:25.347100 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2751c-6e93-4e43-a2d3-701bb1557692 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:25.347398 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2751c-6e93-4e43-a2d3-701bb1557692 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:25.347685 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2751c-6e93-4e43-a2d3-701bb1557692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:25.373827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-79e2751c-6e93-4e43-a2d3-701bb1557692 None None] MFA Rules not processed for user `d0aa07f92f994c1c933b9ff31623319d`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:25.375092 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-79e2751c-6e93-4e43-a2d3-701bb1557692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:25.437626 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1436/2850] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:26:25.442099 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a864bf-a637-4f1d-b06d-d4349c9c5009 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:25.463285 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:25.504189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a864bf-a637-4f1d-b06d-d4349c9c5009 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:25.508931 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c289a-c2c2-435a-a068-28afa8640e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:25.512960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c289a-c2c2-435a-a068-28afa8640e88 admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:25.512960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c289a-c2c2-435a-a068-28afa8640e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:25.512960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039c289a-c2c2-435a-a068-28afa8640e88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:25.512960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c289a-c2c2-435a-a068-28afa8640e88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:25.512960 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039c289a-c2c2-435a-a068-28afa8640e88 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:25.593974 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1415/2851] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:26:25 2022] POST /identity/v3/projects => generated 375 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:26:25.597860 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2751c-6e93-4e43-a2d3-701bb1557692 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:25.608973 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2751c-6e93-4e43-a2d3-701bb1557692 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:25.666035 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645f4c3-c43b-4f91-8ea1-a745a51456b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:25.666740 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645f4c3-c43b-4f91-8ea1-a745a51456b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:25.667034 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645f4c3-c43b-4f91-8ea1-a745a51456b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:25.667319 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c645f4c3-c43b-4f91-8ea1-a745a51456b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:25.668026 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645f4c3-c43b-4f91-8ea1-a745a51456b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:25.669960 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c645f4c3-c43b-4f91-8ea1-a745a51456b8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:25.768019 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1437/2852] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:26:25 2022] POST /identity/v3/users => generated 294 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 04:26:25.770226 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c289a-c2c2-435a-a068-28afa8640e88 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:25.782063 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039c289a-c2c2-435a-a068-28afa8640e88 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:25.786904 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a3e41-0770-4b81-8a3f-4b6fc57c46fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:25.787574 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a3e41-0770-4b81-8a3f-4b6fc57c46fa admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:25.787865 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a3e41-0770-4b81-8a3f-4b6fc57c46fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:25.788144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a3e41-0770-4b81-8a3f-4b6fc57c46fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:25.788878 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a3e41-0770-4b81-8a3f-4b6fc57c46fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:25.790494 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a3e41-0770-4b81-8a3f-4b6fc57c46fa admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:25.801280 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1416/2853] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:26:25.805907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645f4c3-c43b-4f91-8ea1-a745a51456b8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:25.816247 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c645f4c3-c43b-4f91-8ea1-a745a51456b8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:25.821420 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea921bc-cf6c-4749-a777-1053d2294938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:25.822308 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea921bc-cf6c-4749-a777-1053d2294938 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:25.822628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea921bc-cf6c-4749-a777-1053d2294938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:25.822907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea921bc-cf6c-4749-a777-1053d2294938 admin admin] PATH_INFO: `/v3/projects/629777b95c8044e7a01139deb0a7cb50/users/5545619f5b4949168896493e636304bc/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:25.838223 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea921bc-cf6c-4749-a777-1053d2294938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=629777b95c8044e7a01139deb0a7cb50, user_id=5545619f5b4949168896493e636304bc, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:25.840926 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea921bc-cf6c-4749-a777-1053d2294938 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:25.953523 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1438/2854] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:26:25 2022] PUT /identity/v3/projects/629777b95c8044e7a01139deb0a7cb50/users/5545619f5b4949168896493e636304bc/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:26:25.957514 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a3e41-0770-4b81-8a3f-4b6fc57c46fa admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:25.967866 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a3e41-0770-4b81-8a3f-4b6fc57c46fa admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:26.028942 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c3d80-227e-4731-b35e-af749baf1b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:26.029629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c3d80-227e-4731-b35e-af749baf1b2b admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:26.029919 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c3d80-227e-4731-b35e-af749baf1b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:26.030203 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c3d80-227e-4731-b35e-af749baf1b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:26.030975 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1c3d80-227e-4731-b35e-af749baf1b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:26.033209 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1c3d80-227e-4731-b35e-af749baf1b2b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:26.047332 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1417/2855] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:26:26.051890 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea921bc-cf6c-4749-a777-1053d2294938 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:26.062906 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea921bc-cf6c-4749-a777-1053d2294938 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:26.067503 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5719048-ac43-434b-9230-a4d0ae6553e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:26.069108 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5719048-ac43-434b-9230-a4d0ae6553e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:26.069398 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5719048-ac43-434b-9230-a4d0ae6553e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:26.069679 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5719048-ac43-434b-9230-a4d0ae6553e8 admin admin] PATH_INFO: `/v3/projects/629777b95c8044e7a01139deb0a7cb50/users/5545619f5b4949168896493e636304bc/roles/f0f7e28cbe314921afdfb657b163436f` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:26.071696 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5719048-ac43-434b-9230-a4d0ae6553e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=629777b95c8044e7a01139deb0a7cb50, user_id=5545619f5b4949168896493e636304bc, role_id=f0f7e28cbe314921afdfb657b163436f)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:26.074377 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5719048-ac43-434b-9230-a4d0ae6553e8 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:26.084716 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1439/2856] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:26:26 2022] PUT /identity/v3/projects/629777b95c8044e7a01139deb0a7cb50/users/5545619f5b4949168896493e636304bc/roles/f0f7e28cbe314921afdfb657b163436f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:26:26.095633 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f36597-4b92-430b-b7e3-ca5fda0f3ca7 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:26.095925 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f36597-4b92-430b-b7e3-ca5fda0f3ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:26.096207 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f36597-4b92-430b-b7e3-ca5fda0f3ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:26.122816 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-e6f36597-4b92-430b-b7e3-ca5fda0f3ca7 None None] MFA Rules not processed for user `5545619f5b4949168896493e636304bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:26.124014 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e6f36597-4b92-430b-b7e3-ca5fda0f3ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:26.192752 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1418/2857] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:26:26.203628 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5719048-ac43-434b-9230-a4d0ae6553e8 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:26.205224 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-c5719048-ac43-434b-9230-a4d0ae6553e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:26.246218 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:26.264983 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5719048-ac43-434b-9230-a4d0ae6553e8 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:26.268623 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e7b7f-d415-4375-9765-86e0bf8343b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:26.269299 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e7b7f-d415-4375-9765-86e0bf8343b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:26.269606 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e7b7f-d415-4375-9765-86e0bf8343b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:26.269888 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e7b7f-d415-4375-9765-86e0bf8343b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:26.447748 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e7b7f-d415-4375-9765-86e0bf8343b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:26.449159 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e7b7f-d415-4375-9765-86e0bf8343b8 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:26.457907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1440/2858] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:26:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:26:29.394528 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f36597-4b92-430b-b7e3-ca5fda0f3ca7 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:29.412684 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:29.457006 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f36597-4b92-430b-b7e3-ca5fda0f3ca7 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:29.462641 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70225a57-7f51-48dd-a583-80a7cabc6a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:29.463582 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70225a57-7f51-48dd-a583-80a7cabc6a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:29.463868 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70225a57-7f51-48dd-a583-80a7cabc6a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:29.464144 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70225a57-7f51-48dd-a583-80a7cabc6a8b admin admin] PATH_INFO: `/v3/users/d0aa07f92f994c1c933b9ff31623319d` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:29.465343 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70225a57-7f51-48dd-a583-80a7cabc6a8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0aa07f92f994c1c933b9ff31623319d)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:29.467359 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70225a57-7f51-48dd-a583-80a7cabc6a8b admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:29.681241 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-70225a57-7f51-48dd-a583-80a7cabc6a8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0aa07f92f994c1c933b9ff31623319d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:29.692549 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-70225a57-7f51-48dd-a583-80a7cabc6a8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0aa07f92f994c1c933b9ff31623319d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:29.781792 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-70225a57-7f51-48dd-a583-80a7cabc6a8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0aa07f92f994c1c933b9ff31623319d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:29.782459 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-70225a57-7f51-48dd-a583-80a7cabc6a8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0aa07f92f994c1c933b9ff31623319d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:29.783070 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-70225a57-7f51-48dd-a583-80a7cabc6a8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0aa07f92f994c1c933b9ff31623319d'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:29.797515 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1419/2859] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:26:29 2022] DELETE /identity/v3/users/d0aa07f92f994c1c933b9ff31623319d => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:26:29.802073 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b342fe-820e-4b68-98dc-f490d9ebf3df None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:29.802471 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b342fe-820e-4b68-98dc-f490d9ebf3df None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:29.802577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b342fe-820e-4b68-98dc-f490d9ebf3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:29.820352 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-d7b342fe-820e-4b68-98dc-f490d9ebf3df None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:29.821709 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d7b342fe-820e-4b68-98dc-f490d9ebf3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:29.886882 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1441/2860] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:26:29.896392 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70225a57-7f51-48dd-a583-80a7cabc6a8b admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:29.897854 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-70225a57-7f51-48dd-a583-80a7cabc6a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:29.957574 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70225a57-7f51-48dd-a583-80a7cabc6a8b admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:29.961322 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60bff3e-7b0b-4ec2-a3cf-6f8a23acb0ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:29.963037 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bff3e-7b0b-4ec2-a3cf-6f8a23acb0ef service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:29.963248 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bff3e-7b0b-4ec2-a3cf-6f8a23acb0ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:29.963531 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bff3e-7b0b-4ec2-a3cf-6f8a23acb0ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:29.974799 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60bff3e-7b0b-4ec2-a3cf-6f8a23acb0ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:29.976317 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60bff3e-7b0b-4ec2-a3cf-6f8a23acb0ef service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:29.986864 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1420/2861] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:26:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:26:30.405661 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b342fe-820e-4b68-98dc-f490d9ebf3df None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:30.407206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-d7b342fe-820e-4b68-98dc-f490d9ebf3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:30.419546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b342fe-820e-4b68-98dc-f490d9ebf3df None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:30.424955 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579af70-82fe-4ef0-9bb6-93b41a12863c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:30.425855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579af70-82fe-4ef0-9bb6-93b41a12863c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:30.426144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579af70-82fe-4ef0-9bb6-93b41a12863c admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:30.426457 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f579af70-82fe-4ef0-9bb6-93b41a12863c admin admin] PATH_INFO: `/v3/projects/c07ef3d97ff54852b43087b7d223a6a0` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:30.427656 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579af70-82fe-4ef0-9bb6-93b41a12863c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c07ef3d97ff54852b43087b7d223a6a0)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:30.429570 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f579af70-82fe-4ef0-9bb6-93b41a12863c admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:30.566619 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f579af70-82fe-4ef0-9bb6-93b41a12863c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c07ef3d97ff54852b43087b7d223a6a0', 'request_id': 'req-f579af70-82fe-4ef0-9bb6-93b41a12863c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:30.574343 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f579af70-82fe-4ef0-9bb6-93b41a12863c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c07ef3d97ff54852b43087b7d223a6a0', 'request_id': 'req-f579af70-82fe-4ef0-9bb6-93b41a12863c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:30.584046 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f579af70-82fe-4ef0-9bb6-93b41a12863c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c07ef3d97ff54852b43087b7d223a6a0', 'request_id': 'req-f579af70-82fe-4ef0-9bb6-93b41a12863c'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:30.651437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f579af70-82fe-4ef0-9bb6-93b41a12863c admin admin] The token cache is being invalidate because project c07ef3d97ff54852b43087b7d223a6a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:26:30.651741 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f579af70-82fe-4ef0-9bb6-93b41a12863c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:30.652406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-f579af70-82fe-4ef0-9bb6-93b41a12863c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:30.654032 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1442/2862] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:26:30 2022] DELETE /identity/v3/projects/c07ef3d97ff54852b43087b7d223a6a0 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:26:30.658132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60bff3e-7b0b-4ec2-a3cf-6f8a23acb0ef service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:30.668990 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60bff3e-7b0b-4ec2-a3cf-6f8a23acb0ef service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:30.706214 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:30.730203 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d3885-7b06-48a0-a630-62b04e983dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:30.731204 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d3885-7b06-48a0-a630-62b04e983dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:30.731560 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d3885-7b06-48a0-a630-62b04e983dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:30.731910 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d3885-7b06-48a0-a630-62b04e983dfd admin admin] PATH_INFO: `/v3/users/5545619f5b4949168896493e636304bc` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:30.733187 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d3885-7b06-48a0-a630-62b04e983dfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5545619f5b4949168896493e636304bc)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:30.735230 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4d3885-7b06-48a0-a630-62b04e983dfd admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:30.945146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5e4d3885-7b06-48a0-a630-62b04e983dfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5545619f5b4949168896493e636304bc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:30.957669 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5e4d3885-7b06-48a0-a630-62b04e983dfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5545619f5b4949168896493e636304bc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:31.033085 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5e4d3885-7b06-48a0-a630-62b04e983dfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5545619f5b4949168896493e636304bc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:31.033750 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5e4d3885-7b06-48a0-a630-62b04e983dfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5545619f5b4949168896493e636304bc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:31.034375 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-5e4d3885-7b06-48a0-a630-62b04e983dfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5545619f5b4949168896493e636304bc'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:31.047272 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1421/2863] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:26:30 2022] DELETE /identity/v3/users/5545619f5b4949168896493e636304bc => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:26:31.447822 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579af70-82fe-4ef0-9bb6-93b41a12863c admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:31.449566 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-f579af70-82fe-4ef0-9bb6-93b41a12863c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:31.467221 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f579af70-82fe-4ef0-9bb6-93b41a12863c admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:31.481439 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:31.530200 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db2de6-c3db-40da-b85d-4efb30a1bcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:31.531259 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db2de6-c3db-40da-b85d-4efb30a1bcd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:31.531546 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db2de6-c3db-40da-b85d-4efb30a1bcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:31.531854 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0db2de6-c3db-40da-b85d-4efb30a1bcd2 admin admin] PATH_INFO: `/v3/projects/629777b95c8044e7a01139deb0a7cb50` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:31.533203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db2de6-c3db-40da-b85d-4efb30a1bcd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=629777b95c8044e7a01139deb0a7cb50)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:31.535385 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0db2de6-c3db-40da-b85d-4efb30a1bcd2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:31.645159 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e0db2de6-c3db-40da-b85d-4efb30a1bcd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '629777b95c8044e7a01139deb0a7cb50', 'request_id': 'req-e0db2de6-c3db-40da-b85d-4efb30a1bcd2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:31.652464 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e0db2de6-c3db-40da-b85d-4efb30a1bcd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '629777b95c8044e7a01139deb0a7cb50', 'request_id': 'req-e0db2de6-c3db-40da-b85d-4efb30a1bcd2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:31.658402 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e0db2de6-c3db-40da-b85d-4efb30a1bcd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '629777b95c8044e7a01139deb0a7cb50', 'request_id': 'req-e0db2de6-c3db-40da-b85d-4efb30a1bcd2'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:31.730535 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e0db2de6-c3db-40da-b85d-4efb30a1bcd2 admin admin] The token cache is being invalidate because project 629777b95c8044e7a01139deb0a7cb50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80599) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:26:31.730770 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e0db2de6-c3db-40da-b85d-4efb30a1bcd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:31.731368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-e0db2de6-c3db-40da-b85d-4efb30a1bcd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:31.732899 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1443/2864] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:26:31 2022] DELETE /identity/v3/projects/629777b95c8044e7a01139deb0a7cb50 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 04:26:41.043691 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:41.726686 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:51.583899 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f629b5-1082-42e8-b601-a290f2a5940c None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:51.584408 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f629b5-1082-42e8-b601-a290f2a5940c None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:51.584513 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f629b5-1082-42e8-b601-a290f2a5940c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:51.604370 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-d8f629b5-1082-42e8-b601-a290f2a5940c None None] MFA Rules not processed for user `d1cee676e3dc47e18e94de7e20b59455`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:51.605565 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-d8f629b5-1082-42e8-b601-a290f2a5940c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:51.660105 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1422/2865] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:26:55.959425 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db2de6-c3db-40da-b85d-4efb30a1bcd2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:55.961075 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e0db2de6-c3db-40da-b85d-4efb30a1bcd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:56.034174 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0db2de6-c3db-40da-b85d-4efb30a1bcd2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:56.041413 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a631e1-7c03-4646-a0e2-21ab285b68ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:56.042392 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a631e1-7c03-4646-a0e2-21ab285b68ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:56.042950 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a631e1-7c03-4646-a0e2-21ab285b68ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:56.043313 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a631e1-7c03-4646-a0e2-21ab285b68ab admin admin] PATH_INFO: `/v3/users/2b07b50c71b2412183f8683730506e03` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:56.045325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a631e1-7c03-4646-a0e2-21ab285b68ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b07b50c71b2412183f8683730506e03)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:56.047561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a631e1-7c03-4646-a0e2-21ab285b68ab admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:56.299051 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-49a631e1-7c03-4646-a0e2-21ab285b68ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b07b50c71b2412183f8683730506e03'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:56.311514 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-49a631e1-7c03-4646-a0e2-21ab285b68ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b07b50c71b2412183f8683730506e03'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:56.313869 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-49a631e1-7c03-4646-a0e2-21ab285b68ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b07b50c71b2412183f8683730506e03'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:56.313869 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-49a631e1-7c03-4646-a0e2-21ab285b68ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b07b50c71b2412183f8683730506e03'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:56.320050 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-49a631e1-7c03-4646-a0e2-21ab285b68ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b07b50c71b2412183f8683730506e03'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:56.371922 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1444/2866] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:26:55 2022] DELETE /identity/v3/users/2b07b50c71b2412183f8683730506e03 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:26:56.372111 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcbe1c-cc2d-428a-adbe-3f194c2f8be6 None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:56.372111 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcbe1c-cc2d-428a-adbe-3f194c2f8be6 None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:56.372661 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcbe1c-cc2d-428a-adbe-3f194c2f8be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:56.398237 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-37dcbe1c-cc2d-428a-adbe-3f194c2f8be6 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:56.399488 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-37dcbe1c-cc2d-428a-adbe-3f194c2f8be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:56.471027 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1423/2867] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:26:56.480845 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a631e1-7c03-4646-a0e2-21ab285b68ab admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:56.482343 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-49a631e1-7c03-4646-a0e2-21ab285b68ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:56.553074 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a631e1-7c03-4646-a0e2-21ab285b68ab admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:56.557563 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe01b17-e8ae-4cfc-9481-fb3da21f6b40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:56.558300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe01b17-e8ae-4cfc-9481-fb3da21f6b40 service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:56.558686 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe01b17-e8ae-4cfc-9481-fb3da21f6b40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:56.558969 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe01b17-e8ae-4cfc-9481-fb3da21f6b40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:56.569630 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe01b17-e8ae-4cfc-9481-fb3da21f6b40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:56.571112 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe01b17-e8ae-4cfc-9481-fb3da21f6b40 service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:56.584317 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1445/2868] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:26:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:26:57.194708 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcbe1c-cc2d-428a-adbe-3f194c2f8be6 None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:57.196324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-37dcbe1c-cc2d-428a-adbe-3f194c2f8be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:57.213442 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcbe1c-cc2d-428a-adbe-3f194c2f8be6 None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:57.231259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24cecc-760b-48cc-8a29-3267c432f9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:57.232263 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24cecc-760b-48cc-8a29-3267c432f9e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:57.232711 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24cecc-760b-48cc-8a29-3267c432f9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:57.232990 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24cecc-760b-48cc-8a29-3267c432f9e1 admin admin] PATH_INFO: `/v3/projects/ffaa6d4e4e66434792fbb65b9a2a2075` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:57.234512 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24cecc-760b-48cc-8a29-3267c432f9e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffaa6d4e4e66434792fbb65b9a2a2075)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:57.236732 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad24cecc-760b-48cc-8a29-3267c432f9e1 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:57.405223 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad24cecc-760b-48cc-8a29-3267c432f9e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffaa6d4e4e66434792fbb65b9a2a2075', 'request_id': 'req-ad24cecc-760b-48cc-8a29-3267c432f9e1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:57.488609 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad24cecc-760b-48cc-8a29-3267c432f9e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffaa6d4e4e66434792fbb65b9a2a2075', 'request_id': 'req-ad24cecc-760b-48cc-8a29-3267c432f9e1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:57.494413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad24cecc-760b-48cc-8a29-3267c432f9e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffaa6d4e4e66434792fbb65b9a2a2075', 'request_id': 'req-ad24cecc-760b-48cc-8a29-3267c432f9e1'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:57.506649 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad24cecc-760b-48cc-8a29-3267c432f9e1 admin admin] The token cache is being invalidate because project ffaa6d4e4e66434792fbb65b9a2a2075 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:26:57.511789 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad24cecc-760b-48cc-8a29-3267c432f9e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:57.511789 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-ad24cecc-760b-48cc-8a29-3267c432f9e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:26:57.511789 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1424/2869] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:26:57 2022] DELETE /identity/v3/projects/ffaa6d4e4e66434792fbb65b9a2a2075 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:26:57.514840 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db916516-6a80-4a90-a766-49772f6433c8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:57.515311 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db916516-6a80-4a90-a766-49772f6433c8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:57.515420 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db916516-6a80-4a90-a766-49772f6433c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:57.534210 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-db916516-6a80-4a90-a766-49772f6433c8 None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:57.539889 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-db916516-6a80-4a90-a766-49772f6433c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:57.551944 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:57.594223 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1446/2870] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:26:57.602504 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab7ba3-1cfa-4f46-8e3d-1823e22007bb None None] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:57.602807 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab7ba3-1cfa-4f46-8e3d-1823e22007bb None None] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:57.603093 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab7ba3-1cfa-4f46-8e3d-1823e22007bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:57.621468 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.auth.core [None req-f1ab7ba3-1cfa-4f46-8e3d-1823e22007bb None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80600) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:57.623369 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f1ab7ba3-1cfa-4f46-8e3d-1823e22007bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:57.631199 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1425/2871] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:26:57.635382 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db916516-6a80-4a90-a766-49772f6433c8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:57.645605 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db916516-6a80-4a90-a766-49772f6433c8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:57.655292 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d163c-5a56-4874-b2bc-b7910bc07d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:57.660362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d163c-5a56-4874-b2bc-b7910bc07d68 admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:57.660362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d163c-5a56-4874-b2bc-b7910bc07d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:57.660362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d163c-5a56-4874-b2bc-b7910bc07d68 admin admin] PATH_INFO: `/v3/domains` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:57.660362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d163c-5a56-4874-b2bc-b7910bc07d68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80599) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 04:26:57.660362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d163c-5a56-4874-b2bc-b7910bc07d68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:57.660362 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d163c-5a56-4874-b2bc-b7910bc07d68 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:57.674325 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1447/2872] 10.222.0.16 () {58 vars in 1152 bytes} [Fri Jun 24 04:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 04:26:57.679738 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab7ba3-1cfa-4f46-8e3d-1823e22007bb None None] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:57.691457 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab7ba3-1cfa-4f46-8e3d-1823e22007bb None None] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:57.696173 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275dd11c-4e75-4215-9ad7-d19be67f37dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:57.696861 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dd11c-4e75-4215-9ad7-d19be67f37dd admin admin] REQUEST_METHOD: `POST` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:57.697149 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dd11c-4e75-4215-9ad7-d19be67f37dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:57.697430 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275dd11c-4e75-4215-9ad7-d19be67f37dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:57.698139 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275dd11c-4e75-4215-9ad7-d19be67f37dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:57.700084 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275dd11c-4e75-4215-9ad7-d19be67f37dd admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:57.799889 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:57.806103 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1426/2873] 10.222.0.16 () {60 vars in 1152 bytes} [Fri Jun 24 04:26:57 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:26:57.811698 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d163c-5a56-4874-b2bc-b7910bc07d68 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:57.820894 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d163c-5a56-4874-b2bc-b7910bc07d68 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:57.876969 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc82ab-fa4f-4786-b2e3-e547a7055a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:57.877646 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc82ab-fa4f-4786-b2e3-e547a7055a42 admin admin] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:57.877949 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc82ab-fa4f-4786-b2e3-e547a7055a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:57.878215 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc82ab-fa4f-4786-b2e3-e547a7055a42 admin admin] PATH_INFO: `/v3/users` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:57.879003 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc82ab-fa4f-4786-b2e3-e547a7055a42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:57.880907 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc82ab-fa4f-4786-b2e3-e547a7055a42 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:58.021924 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1448/2874] 10.222.0.16 () {60 vars in 1143 bytes} [Fri Jun 24 04:26:57 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 04:26:58.026407 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275dd11c-4e75-4215-9ad7-d19be67f37dd admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:58.037783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275dd11c-4e75-4215-9ad7-d19be67f37dd admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:58.042354 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bed697-0e0d-4290-a8c7-0b84c2584e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:58.043377 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bed697-0e0d-4290-a8c7-0b84c2584e13 admin admin] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:58.043377 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bed697-0e0d-4290-a8c7-0b84c2584e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:58.043587 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bed697-0e0d-4290-a8c7-0b84c2584e13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:58.044303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bed697-0e0d-4290-a8c7-0b84c2584e13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:58.046608 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bed697-0e0d-4290-a8c7-0b84c2584e13 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:58.057270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1427/2875] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:26:58.061331 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc82ab-fa4f-4786-b2e3-e547a7055a42 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:58.071430 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc82ab-fa4f-4786-b2e3-e547a7055a42 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:58.076156 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47944727-3d66-4eac-9f29-19a30cb6476d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:58.076818 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944727-3d66-4eac-9f29-19a30cb6476d admin admin] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:58.077110 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944727-3d66-4eac-9f29-19a30cb6476d admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:58.077393 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47944727-3d66-4eac-9f29-19a30cb6476d admin admin] PATH_INFO: `/v3/roles` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:58.078122 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47944727-3d66-4eac-9f29-19a30cb6476d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:58.079871 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47944727-3d66-4eac-9f29-19a30cb6476d admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:58.089725 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1449/2876] 10.222.0.16 () {58 vars in 1121 bytes} [Fri Jun 24 04:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 04:26:58.094928 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bed697-0e0d-4290-a8c7-0b84c2584e13 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:58.104710 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bed697-0e0d-4290-a8c7-0b84c2584e13 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:58.110475 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509ecba-442e-49af-9c9c-d3a43d84da32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:58.111377 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509ecba-442e-49af-9c9c-d3a43d84da32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:58.111658 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509ecba-442e-49af-9c9c-d3a43d84da32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:58.111934 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509ecba-442e-49af-9c9c-d3a43d84da32 admin admin] PATH_INFO: `/v3/projects/4a2dfcc0f1ad4daf8b26339e3eb51c53/users/fc82a6133e284d8e97a7a6f0a6c73f24/roles/d7aadc20cd4c4838ae5d7d8e8826abae` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:58.126912 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509ecba-442e-49af-9c9c-d3a43d84da32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a2dfcc0f1ad4daf8b26339e3eb51c53, user_id=fc82a6133e284d8e97a7a6f0a6c73f24, role_id=d7aadc20cd4c4838ae5d7d8e8826abae)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:58.129545 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509ecba-442e-49af-9c9c-d3a43d84da32 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:58.205314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1428/2877] 10.222.0.16 () {60 vars in 1482 bytes} [Fri Jun 24 04:26:58 2022] PUT /identity/v3/projects/4a2dfcc0f1ad4daf8b26339e3eb51c53/users/fc82a6133e284d8e97a7a6f0a6c73f24/roles/d7aadc20cd4c4838ae5d7d8e8826abae => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:26:58.210499 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45250812-6844-4f03-8db3-bce54576c2ac None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:58.210797 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45250812-6844-4f03-8db3-bce54576c2ac None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:58.211109 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45250812-6844-4f03-8db3-bce54576c2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:58.229882 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-45250812-6844-4f03-8db3-bce54576c2ac None None] MFA Rules not processed for user `c531d2732de94f47a66f16db7047917f`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:58.231138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-45250812-6844-4f03-8db3-bce54576c2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:58.292300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1450/2878] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 04:26:58.303565 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509ecba-442e-49af-9c9c-d3a43d84da32 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:58.305384 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-3509ecba-442e-49af-9c9c-d3a43d84da32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:58.364461 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509ecba-442e-49af-9c9c-d3a43d84da32 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:58.368155 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25b362-93d6-42f1-a805-7091be77c3f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:58.368851 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25b362-93d6-42f1-a805-7091be77c3f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:58.369142 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25b362-93d6-42f1-a805-7091be77c3f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:58.369426 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25b362-93d6-42f1-a805-7091be77c3f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:58.378846 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25b362-93d6-42f1-a805-7091be77c3f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:58.380327 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25b362-93d6-42f1-a805-7091be77c3f0 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:58.392023 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1429/2879] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 04:26:59.708531 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9098c9-f913-4f34-997c-c2f0d7685ced None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:59.709078 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9098c9-f913-4f34-997c-c2f0d7685ced None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:59.709191 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9098c9-f913-4f34-997c-c2f0d7685ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:59.739107 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-6d9098c9-f913-4f34-997c-c2f0d7685ced None None] MFA Rules not processed for user `fc82a6133e284d8e97a7a6f0a6c73f24`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:26:59.740373 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-6d9098c9-f913-4f34-997c-c2f0d7685ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:59.759083 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:26:59.806823 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1451/2880] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:26:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:26:59.819021 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25b362-93d6-42f1-a805-7091be77c3f0 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:26:59.820413 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-fa25b362-93d6-42f1-a805-7091be77c3f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:26:59.877498 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25b362-93d6-42f1-a805-7091be77c3f0 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:26:59.881324 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd5521-88e8-40d8-bcfd-954e83397ecd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:26:59.881962 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd5521-88e8-40d8-bcfd-954e83397ecd service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:26:59.882308 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd5521-88e8-40d8-bcfd-954e83397ecd service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:26:59.882620 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd5521-88e8-40d8-bcfd-954e83397ecd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:26:59.891089 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd5521-88e8-40d8-bcfd-954e83397ecd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:26:59.892506 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd5521-88e8-40d8-bcfd-954e83397ecd service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:26:59.901554 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1430/2881] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:26:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:27:01.596677 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccf3de-9dd0-4356-8154-ae1d48c1c2c1 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:27:01.597206 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccf3de-9dd0-4356-8154-ae1d48c1c2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:27:01.597319 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccf3de-9dd0-4356-8154-ae1d48c1c2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:27:01.615133 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-deccf3de-9dd0-4356-8154-ae1d48c1c2c1 None None] MFA Rules not processed for user `fc82a6133e284d8e97a7a6f0a6c73f24`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:27:01.616321 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-deccf3de-9dd0-4356-8154-ae1d48c1c2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:27:01.624012 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1452/2882] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:27:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:27:09.897299 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=67,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:27:11.610153 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=22 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:27:15.065691 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd5521-88e8-40d8-bcfd-954e83397ecd service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:27:15.067148 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-adcd5521-88e8-40d8-bcfd-954e83397ecd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:27:15.080056 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd5521-88e8-40d8-bcfd-954e83397ecd service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:27:15.083633 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c9e62-461b-4af0-9802-2778e6f2d623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:27:15.084298 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c9e62-461b-4af0-9802-2778e6f2d623 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:27:15.084580 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c9e62-461b-4af0-9802-2778e6f2d623 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:27:15.084860 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c9e62-461b-4af0-9802-2778e6f2d623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:27:15.086619 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2d1c9e62-461b-4af0-9802-2778e6f2d623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:27:15.157417 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c9e62-461b-4af0-9802-2778e6f2d623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:27:15.158916 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c9e62-461b-4af0-9802-2778e6f2d623 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:27:15.170221 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1431/2883] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:27:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:27:25.165162 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=34 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:27:30.617067 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccf3de-9dd0-4356-8154-ae1d48c1c2c1 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:27:30.618561 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-deccf3de-9dd0-4356-8154-ae1d48c1c2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:27:30.631085 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccf3de-9dd0-4356-8154-ae1d48c1c2c1 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:27:30.638823 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a31889-6363-4549-9325-cd924856b030 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:27:30.638823 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a31889-6363-4549-9325-cd924856b030 service cinder] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:27:30.638823 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a31889-6363-4549-9325-cd924856b030 service cinder] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:27:30.638823 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a31889-6363-4549-9325-cd924856b030 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:27:30.639259 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-21a31889-6363-4549-9325-cd924856b030 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:27:30.650862 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a31889-6363-4549-9325-cd924856b030 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:27:30.652531 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a31889-6363-4549-9325-cd924856b030 service cinder] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:27:30.663855 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1453/2884] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:27:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:27:40.658520 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=6 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:28:01.283486 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c9e62-461b-4af0-9802-2778e6f2d623 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:28:01.284949 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-2d1c9e62-461b-4af0-9802-2778e6f2d623 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:28:01.299148 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c9e62-461b-4af0-9802-2778e6f2d623 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:28:01.305400 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ac8da-2da3-45a7-b9f9-f52f92f61f67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:28:01.312314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ac8da-2da3-45a7-b9f9-f52f92f61f67 service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:28:01.312314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ac8da-2da3-45a7-b9f9-f52f92f61f67 service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:28:01.312314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ac8da-2da3-45a7-b9f9-f52f92f61f67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:28:01.312314 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-f41ac8da-2da3-45a7-b9f9-f52f92f61f67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:28:01.320955 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ac8da-2da3-45a7-b9f9-f52f92f61f67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:28:01.322560 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ac8da-2da3-45a7-b9f9-f52f92f61f67 service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:28:01.333474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1432/2885] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:28:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:28:11.328689 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=6 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:28:13.629052 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44616864-7fe7-40f6-9e23-a8ef89c8dad8 None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:28:13.630057 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44616864-7fe7-40f6-9e23-a8ef89c8dad8 None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:28:13.630186 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44616864-7fe7-40f6-9e23-a8ef89c8dad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:28:13.647843 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-44616864-7fe7-40f6-9e23-a8ef89c8dad8 None None] MFA Rules not processed for user `fc82a6133e284d8e97a7a6f0a6c73f24`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:28:13.649044 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-44616864-7fe7-40f6-9e23-a8ef89c8dad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:28:13.656153 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1454/2886] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:28:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:28:13.884930 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ac8da-2da3-45a7-b9f9-f52f92f61f67 service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:28:13.894186 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ac8da-2da3-45a7-b9f9-f52f92f61f67 service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:28:13.898146 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573d340-a368-40a8-93f7-8b63cb591a5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:28:13.898802 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573d340-a368-40a8-93f7-8b63cb591a5d service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:28:13.899086 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573d340-a368-40a8-93f7-8b63cb591a5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:28:13.899388 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573d340-a368-40a8-93f7-8b63cb591a5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:28:13.901140 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e573d340-a368-40a8-93f7-8b63cb591a5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:28:13.910958 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573d340-a368-40a8-93f7-8b63cb591a5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:28:13.912391 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573d340-a368-40a8-93f7-8b63cb591a5d service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:28:13.920627 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1433/2887] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:28:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:28:14.523412 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44616864-7fe7-40f6-9e23-a8ef89c8dad8 None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:28:14.524965 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-44616864-7fe7-40f6-9e23-a8ef89c8dad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:28:14.596219 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44616864-7fe7-40f6-9e23-a8ef89c8dad8 None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:28:14.601741 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124303c7-5258-4fba-b31d-b325edbf1f24 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7662e61619d44c92af3ffacbacba3913', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:28:14.602433 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124303c7-5258-4fba-b31d-b325edbf1f24 service placement] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:28:14.602720 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124303c7-5258-4fba-b31d-b325edbf1f24 service placement] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:28:14.602997 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124303c7-5258-4fba-b31d-b325edbf1f24 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:28:14.604762 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-124303c7-5258-4fba-b31d-b325edbf1f24 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:28:14.616538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124303c7-5258-4fba-b31d-b325edbf1f24 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:28:14.618103 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124303c7-5258-4fba-b31d-b325edbf1f24 service placement] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:28:14.627961 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1455/2888] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:28:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:28:17.377690 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573d340-a368-40a8-93f7-8b63cb591a5d service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:28:17.387859 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573d340-a368-40a8-93f7-8b63cb591a5d service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:28:17.391563 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f597b-3ef7-41f7-975e-8e7115b99156 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:28:17.393462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f597b-3ef7-41f7-975e-8e7115b99156 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:28:17.393462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f597b-3ef7-41f7-975e-8e7115b99156 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:28:17.393462 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f597b-3ef7-41f7-975e-8e7115b99156 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:28:17.394637 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-e85f597b-3ef7-41f7-975e-8e7115b99156 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:28:17.405551 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f597b-3ef7-41f7-975e-8e7115b99156 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:28:17.407015 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f597b-3ef7-41f7-975e-8e7115b99156 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:28:17.415937 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1434/2889] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:28:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:28:22.804557 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124303c7-5258-4fba-b31d-b325edbf1f24 service placement] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:28:22.806028 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-124303c7-5258-4fba-b31d-b325edbf1f24 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:28:22.821849 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124303c7-5258-4fba-b31d-b325edbf1f24 service placement] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:28:22.827515 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d5ac07-4245-4e3c-92e5-456c6c4b0f45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:28:22.828750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5ac07-4245-4e3c-92e5-456c6c4b0f45 service nova] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:28:22.828750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5ac07-4245-4e3c-92e5-456c6c4b0f45 service nova] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:28:22.828750 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d5ac07-4245-4e3c-92e5-456c6c4b0f45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:28:22.830642 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-e0d5ac07-4245-4e3c-92e5-456c6c4b0f45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:28:22.843826 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d5ac07-4245-4e3c-92e5-456c6c4b0f45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:28:22.845409 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d5ac07-4245-4e3c-92e5-456c6c4b0f45 service nova] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:28:22.856820 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1456/2890] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:28:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:28:27.411420 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=12 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:28:32.852226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=41 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:28:48.387359 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f597b-3ef7-41f7-975e-8e7115b99156 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:28:48.398628 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f597b-3ef7-41f7-975e-8e7115b99156 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:28:48.402618 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8ea7c-8b4b-416a-b9c8-a1703c06f220 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fbec7435e0841a5b96946dca814e0c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:28:48.403263 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8ea7c-8b4b-416a-b9c8-a1703c06f220 service cinder] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:28:48.403545 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8ea7c-8b4b-416a-b9c8-a1703c06f220 service cinder] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:28:48.403823 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8ea7c-8b4b-416a-b9c8-a1703c06f220 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:28:48.405592 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5ed8ea7c-8b4b-416a-b9c8-a1703c06f220 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:28:48.466711 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8ea7c-8b4b-416a-b9c8-a1703c06f220 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:28:48.468150 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8ea7c-8b4b-416a-b9c8-a1703c06f220 service cinder] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:28:48.478437 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1435/2891] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:28:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:28:56.914263 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f21e65-aa4a-4a6f-b50e-bd341b2783ea None None] REQUEST_METHOD: `POST` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:28:56.914833 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f21e65-aa4a-4a6f-b50e-bd341b2783ea None None] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:28:56.914953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f21e65-aa4a-4a6f-b50e-bd341b2783ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:28:56.933030 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.auth.core [None req-95f21e65-aa4a-4a6f-b50e-bd341b2783ea None None] MFA Rules not processed for user `d1cee676e3dc47e18e94de7e20b59455`. Rule list: `[]` (Enabled: `True`). {{(pid=80599) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 04:28:56.934220 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-95f21e65-aa4a-4a6f-b50e-bd341b2783ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:28:56.940964 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1457/2892] 10.222.0.16 () {58 vars in 957 bytes} [Fri Jun 24 04:28:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 04:28:58.473132 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=28 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:29:06.926771 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=3 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:29:15.447197 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8ea7c-8b4b-416a-b9c8-a1703c06f220 service cinder] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:29:15.456581 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8ea7c-8b4b-416a-b9c8-a1703c06f220 service cinder] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:29:15.461135 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421d0c9-9fdb-48d0-abd5-32cd09e5fde1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:29:15.461690 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421d0c9-9fdb-48d0-abd5-32cd09e5fde1 service neutron] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:29:15.461978 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421d0c9-9fdb-48d0-abd5-32cd09e5fde1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:29:15.462257 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421d0c9-9fdb-48d0-abd5-32cd09e5fde1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:29:15.464178 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-5421d0c9-9fdb-48d0-abd5-32cd09e5fde1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:29:15.527490 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421d0c9-9fdb-48d0-abd5-32cd09e5fde1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:29:15.528914 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421d0c9-9fdb-48d0-abd5-32cd09e5fde1 service neutron] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:29:15.538621 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1436/2893] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:29:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:29:21.739735 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f21e65-aa4a-4a6f-b50e-bd341b2783ea None None] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:29:21.751033 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f21e65-aa4a-4a6f-b50e-bd341b2783ea None None] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:29:21.755503 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c85346-6f91-4786-9f0b-a4c611c151f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:29:21.756368 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c85346-6f91-4786-9f0b-a4c611c151f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:29:21.756651 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c85346-6f91-4786-9f0b-a4c611c151f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:29:21.756930 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c85346-6f91-4786-9f0b-a4c611c151f2 admin admin] PATH_INFO: `/v3/users/fc82a6133e284d8e97a7a6f0a6c73f24` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:29:21.758674 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c85346-6f91-4786-9f0b-a4c611c151f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc82a6133e284d8e97a7a6f0a6c73f24)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:29:21.760778 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c85346-6f91-4786-9f0b-a4c611c151f2 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:29:21.929042 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-52c85346-6f91-4786-9f0b-a4c611c151f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc82a6133e284d8e97a7a6f0a6c73f24'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:29:21.943552 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-52c85346-6f91-4786-9f0b-a4c611c151f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc82a6133e284d8e97a7a6f0a6c73f24'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:29:21.944681 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-52c85346-6f91-4786-9f0b-a4c611c151f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc82a6133e284d8e97a7a6f0a6c73f24'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:29:21.945322 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-52c85346-6f91-4786-9f0b-a4c611c151f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc82a6133e284d8e97a7a6f0a6c73f24'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:29:21.953068 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-52c85346-6f91-4786-9f0b-a4c611c151f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc82a6133e284d8e97a7a6f0a6c73f24'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:29:22.023803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1458/2894] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:29:21 2022] DELETE /identity/v3/users/fc82a6133e284d8e97a7a6f0a6c73f24 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:29:22.278307 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421d0c9-9fdb-48d0-abd5-32cd09e5fde1 service neutron] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:29:22.288303 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421d0c9-9fdb-48d0-abd5-32cd09e5fde1 service neutron] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:29:22.350626 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493d729-7a04-414f-8895-159163264e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:29:22.351471 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493d729-7a04-414f-8895-159163264e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:29:22.352254 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493d729-7a04-414f-8895-159163264e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:29:22.352537 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493d729-7a04-414f-8895-159163264e33 admin admin] PATH_INFO: `/v3/projects/4a2dfcc0f1ad4daf8b26339e3eb51c53` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:29:22.353729 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493d729-7a04-414f-8895-159163264e33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a2dfcc0f1ad4daf8b26339e3eb51c53)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:29:22.355763 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493d729-7a04-414f-8895-159163264e33 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:29:22.441690 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8493d729-7a04-414f-8895-159163264e33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a2dfcc0f1ad4daf8b26339e3eb51c53', 'request_id': 'req-8493d729-7a04-414f-8895-159163264e33'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:29:22.503557 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8493d729-7a04-414f-8895-159163264e33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a2dfcc0f1ad4daf8b26339e3eb51c53', 'request_id': 'req-8493d729-7a04-414f-8895-159163264e33'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:29:22.509710 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8493d729-7a04-414f-8895-159163264e33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a2dfcc0f1ad4daf8b26339e3eb51c53', 'request_id': 'req-8493d729-7a04-414f-8895-159163264e33'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:29:22.520745 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8493d729-7a04-414f-8895-159163264e33 admin admin] The token cache is being invalidate because project 4a2dfcc0f1ad4daf8b26339e3eb51c53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:29:22.520978 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8493d729-7a04-414f-8895-159163264e33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:29:22.521622 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-8493d729-7a04-414f-8895-159163264e33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:29:22.523259 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1437/2895] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:29:22 2022] DELETE /identity/v3/projects/4a2dfcc0f1ad4daf8b26339e3eb51c53 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:29:32.020216 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:29:32.517519 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:30:27.879847 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c85346-6f91-4786-9f0b-a4c611c151f2 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:30:27.881406 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-52c85346-6f91-4786-9f0b-a4c611c151f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:30:28.034097 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c85346-6f91-4786-9f0b-a4c611c151f2 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:30:28.101017 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e6175-517f-4dba-b088-0c4d7a87b39e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7662e61619d44c92af3ffacbacba3913', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:30:28.101803 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e6175-517f-4dba-b088-0c4d7a87b39e service placement] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:30:28.102095 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e6175-517f-4dba-b088-0c4d7a87b39e service placement] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:30:28.102389 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e6175-517f-4dba-b088-0c4d7a87b39e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:30:28.104226 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-307e6175-517f-4dba-b088-0c4d7a87b39e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:30:28.116054 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e6175-517f-4dba-b088-0c4d7a87b39e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:30:28.117591 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e6175-517f-4dba-b088-0c4d7a87b39e service placement] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:30:28.128709 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1459/2896] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:30:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:30:38.124625 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=89 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:30:57.827791 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493d729-7a04-414f-8895-159163264e33 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:30:57.829304 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8493d729-7a04-414f-8895-159163264e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:30:57.976278 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493d729-7a04-414f-8895-159163264e33 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:30:57.982176 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fe7ea-4975-4206-aa92-7346aca71c6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80b7b3ec60db4de19bbf7d08f111f470', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:30:57.982869 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fe7ea-4975-4206-aa92-7346aca71c6b service nova] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:30:57.983154 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fe7ea-4975-4206-aa92-7346aca71c6b service nova] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:30:57.983431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fe7ea-4975-4206-aa92-7346aca71c6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:30:57.985202 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8b3fe7ea-4975-4206-aa92-7346aca71c6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:30:57.999556 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fe7ea-4975-4206-aa92-7346aca71c6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:30:58.004305 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fe7ea-4975-4206-aa92-7346aca71c6b service nova] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:30:58.012984 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1438/2897] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:30:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:31:08.009036 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=57 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:31:12.107694 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e6175-517f-4dba-b088-0c4d7a87b39e service placement] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:31:12.109109 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-307e6175-517f-4dba-b088-0c4d7a87b39e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:31:12.185639 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e6175-517f-4dba-b088-0c4d7a87b39e service placement] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:31:12.189204 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964e4c0-70f9-49ec-bd65-0b7a5f71283d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd98b54f587f4b42b4dd0a058b30c684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:31:12.189847 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964e4c0-70f9-49ec-bd65-0b7a5f71283d service neutron] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:31:12.190128 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964e4c0-70f9-49ec-bd65-0b7a5f71283d service neutron] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:31:12.190437 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964e4c0-70f9-49ec-bd65-0b7a5f71283d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:31:12.192228 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a964e4c0-70f9-49ec-bd65-0b7a5f71283d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:31:12.274300 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964e4c0-70f9-49ec-bd65-0b7a5f71283d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:31:12.275695 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964e4c0-70f9-49ec-bd65-0b7a5f71283d service neutron] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:31:12.294827 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1460/2898] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:31:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:31:15.770864 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fe7ea-4975-4206-aa92-7346aca71c6b service nova] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:31:15.772309 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-8b3fe7ea-4975-4206-aa92-7346aca71c6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:31:15.787467 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fe7ea-4975-4206-aa92-7346aca71c6b service nova] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:31:15.792455 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce0fea-fb02-4e20-802d-ff1444e3cb30 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7662e61619d44c92af3ffacbacba3913', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:31:15.793096 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce0fea-fb02-4e20-802d-ff1444e3cb30 service placement] REQUEST_METHOD: `GET` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:31:15.793375 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce0fea-fb02-4e20-802d-ff1444e3cb30 service placement] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:31:15.793651 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce0fea-fb02-4e20-802d-ff1444e3cb30 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:31:15.795404 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-a6ce0fea-fb02-4e20-802d-ff1444e3cb30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:31:15.807545 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce0fea-fb02-4e20-802d-ff1444e3cb30 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:31:15.809156 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce0fea-fb02-4e20-802d-ff1444e3cb30 service placement] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:31:15.819629 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1439/2899] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:31:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:31:19.399953 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964e4c0-70f9-49ec-bd65-0b7a5f71283d service neutron] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:31:19.401443 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-a964e4c0-70f9-49ec-bd65-0b7a5f71283d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:31:19.415211 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964e4c0-70f9-49ec-bd65-0b7a5f71283d service neutron] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:31:19.419494 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfafcb8-90e5-419a-b0b9-a71cf056b584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:31:19.420408 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfafcb8-90e5-419a-b0b9-a71cf056b584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:31:19.420645 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfafcb8-90e5-419a-b0b9-a71cf056b584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:31:19.420918 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfafcb8-90e5-419a-b0b9-a71cf056b584 admin admin] PATH_INFO: `/v3/users/d1cee676e3dc47e18e94de7e20b59455` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:31:19.422088 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfafcb8-90e5-419a-b0b9-a71cf056b584 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1cee676e3dc47e18e94de7e20b59455)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:31:19.424048 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfafcb8-90e5-419a-b0b9-a71cf056b584 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:31:19.596714 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5dfafcb8-90e5-419a-b0b9-a71cf056b584 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1cee676e3dc47e18e94de7e20b59455'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:19.607493 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:31:19.609645 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5dfafcb8-90e5-419a-b0b9-a71cf056b584 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1cee676e3dc47e18e94de7e20b59455'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:19.610347 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5dfafcb8-90e5-419a-b0b9-a71cf056b584 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1cee676e3dc47e18e94de7e20b59455'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:19.611013 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5dfafcb8-90e5-419a-b0b9-a71cf056b584 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1cee676e3dc47e18e94de7e20b59455'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:19.618624 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-5dfafcb8-90e5-419a-b0b9-a71cf056b584 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1cee676e3dc47e18e94de7e20b59455'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:19.697053 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1461/2900] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:31:19 2022] DELETE /identity/v3/users/d1cee676e3dc47e18e94de7e20b59455 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:31:20.018431 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce0fea-fb02-4e20-802d-ff1444e3cb30 service placement] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:31:20.030874 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce0fea-fb02-4e20-802d-ff1444e3cb30 service placement] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:31:20.093839 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86992dab-abfb-4a4a-853c-72586f7b3408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:31:20.094769 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86992dab-abfb-4a4a-853c-72586f7b3408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:31:20.095088 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86992dab-abfb-4a4a-853c-72586f7b3408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:31:20.095368 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86992dab-abfb-4a4a-853c-72586f7b3408 admin admin] PATH_INFO: `/v3/projects/f9c12f0aa786483e8d1ce8ff9042164f` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:31:20.097072 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86992dab-abfb-4a4a-853c-72586f7b3408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9c12f0aa786483e8d1ce8ff9042164f)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:31:20.099330 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86992dab-abfb-4a4a-853c-72586f7b3408 admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:31:20.188656 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-86992dab-abfb-4a4a-853c-72586f7b3408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9c12f0aa786483e8d1ce8ff9042164f', 'request_id': 'req-86992dab-abfb-4a4a-853c-72586f7b3408'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:20.244823 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-86992dab-abfb-4a4a-853c-72586f7b3408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9c12f0aa786483e8d1ce8ff9042164f', 'request_id': 'req-86992dab-abfb-4a4a-853c-72586f7b3408'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:20.250307 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-86992dab-abfb-4a4a-853c-72586f7b3408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9c12f0aa786483e8d1ce8ff9042164f', 'request_id': 'req-86992dab-abfb-4a4a-853c-72586f7b3408'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:20.262690 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-86992dab-abfb-4a4a-853c-72586f7b3408 admin admin] The token cache is being invalidate because project f9c12f0aa786483e8d1ce8ff9042164f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:31:20.262983 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-86992dab-abfb-4a4a-853c-72586f7b3408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:20.263621 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-86992dab-abfb-4a4a-853c-72586f7b3408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:20.265156 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1440/2901] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:31:20 2022] DELETE /identity/v3/projects/f9c12f0aa786483e8d1ce8ff9042164f => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:31:20.269155 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfafcb8-90e5-419a-b0b9-a71cf056b584 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:31:20.270577 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-5dfafcb8-90e5-419a-b0b9-a71cf056b584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:31:20.283828 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfafcb8-90e5-419a-b0b9-a71cf056b584 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:31:20.340539 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c532b-9e86-457c-b6c3-172f55bfc502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:31:20.341430 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c532b-9e86-457c-b6c3-172f55bfc502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:31:20.341715 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c532b-9e86-457c-b6c3-172f55bfc502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:31:20.341990 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c532b-9e86-457c-b6c3-172f55bfc502 admin admin] PATH_INFO: `/v3/users/53baf1b3e4634b25b7f188a9843a4ba1` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:31:20.343248 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c532b-9e86-457c-b6c3-172f55bfc502 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53baf1b3e4634b25b7f188a9843a4ba1)` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:31:20.345161 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188c532b-9e86-457c-b6c3-172f55bfc502 admin admin] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:31:20.488143 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-188c532b-9e86-457c-b6c3-172f55bfc502 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53baf1b3e4634b25b7f188a9843a4ba1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:20.500218 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-188c532b-9e86-457c-b6c3-172f55bfc502 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53baf1b3e4634b25b7f188a9843a4ba1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:20.500841 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-188c532b-9e86-457c-b6c3-172f55bfc502 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53baf1b3e4634b25b7f188a9843a4ba1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:20.501462 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-188c532b-9e86-457c-b6c3-172f55bfc502 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53baf1b3e4634b25b7f188a9843a4ba1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:20.508955 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.notifications [None req-188c532b-9e86-457c-b6c3-172f55bfc502 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53baf1b3e4634b25b7f188a9843a4ba1'} {{(pid=80599) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:20.565138 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1462/2902] 10.222.0.16 () {58 vars in 1223 bytes} [Fri Jun 24 04:31:20 2022] DELETE /identity/v3/users/53baf1b3e4634b25b7f188a9843a4ba1 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:31:20.802193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86992dab-abfb-4a4a-853c-72586f7b3408 admin admin] Authenticating user token {{(pid=80600) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:31:20.803762 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.fernet_utils [None req-86992dab-abfb-4a4a-853c-72586f7b3408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80600) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:31:20.817189 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86992dab-abfb-4a4a-853c-72586f7b3408 admin admin] Validating token access rules against request {{(pid=80600) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:31:20.877945 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd5537-9bed-4b7c-b73e-5f601910fe8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c531d2732de94f47a66f16db7047917f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6a1c0f0b20b844d7b8311813151d93aa', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80600) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:31:20.878902 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd5537-9bed-4b7c-b73e-5f601910fe8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:31:20.879193 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd5537-9bed-4b7c-b73e-5f601910fe8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:31:20.879474 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd5537-9bed-4b7c-b73e-5f601910fe8a admin admin] PATH_INFO: `/v3/projects/3c7828b5281148b89afe2f97d334a613` {{(pid=80600) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:31:20.880705 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd5537-9bed-4b7c-b73e-5f601910fe8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c7828b5281148b89afe2f97d334a613)` {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:31:20.882670 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd5537-9bed-4b7c-b73e-5f601910fe8a admin admin] RBAC: Authorization granted {{(pid=80600) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:31:20.903783 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:31:20.982883 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-39fd5537-9bed-4b7c-b73e-5f601910fe8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c7828b5281148b89afe2f97d334a613', 'request_id': 'req-39fd5537-9bed-4b7c-b73e-5f601910fe8a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:21.039735 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-39fd5537-9bed-4b7c-b73e-5f601910fe8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c7828b5281148b89afe2f97d334a613', 'request_id': 'req-39fd5537-9bed-4b7c-b73e-5f601910fe8a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:21.049662 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-39fd5537-9bed-4b7c-b73e-5f601910fe8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c7828b5281148b89afe2f97d334a613', 'request_id': 'req-39fd5537-9bed-4b7c-b73e-5f601910fe8a'} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:21.058270 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-39fd5537-9bed-4b7c-b73e-5f601910fe8a admin admin] The token cache is being invalidate because project 3c7828b5281148b89afe2f97d334a613 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80600) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 04:31:21.058593 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-39fd5537-9bed-4b7c-b73e-5f601910fe8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:21.059266 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG keystone.notifications [None req-39fd5537-9bed-4b7c-b73e-5f601910fe8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80600) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 04:31:21.060836 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: [pid: 80600|app: 0|req: 1441/2903] 10.222.0.16 () {58 vars in 1232 bytes} [Fri Jun 24 04:31:20 2022] DELETE /identity/v3/projects/3c7828b5281148b89afe2f97d334a613 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 04:31:30.561538 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=62,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:31:31.096453 dv-cinder-846842-3-iscsi devstack@keystone.service[80600]: DEBUG dbcounter [-] [80600] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80600) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 04:35:32.820950 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c532b-9e86-457c-b6c3-172f55bfc502 admin admin] Authenticating user token {{(pid=80599) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 04:35:32.822440 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-188c532b-9e86-457c-b6c3-172f55bfc502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:35:32.898326 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188c532b-9e86-457c-b6c3-172f55bfc502 admin admin] Validating token access rules against request {{(pid=80599) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 04:35:32.960490 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc139e7-ba66-4ff4-9260-f7e9d642f6d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7662e61619d44c92af3ffacbacba3913', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68d376986e854a4bb334676572340b4c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80599) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 04:35:32.961144 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc139e7-ba66-4ff4-9260-f7e9d642f6d5 service placement] REQUEST_METHOD: `GET` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 04:35:32.961435 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc139e7-ba66-4ff4-9260-f7e9d642f6d5 service placement] SCRIPT_NAME: `/identity` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 04:35:32.961715 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc139e7-ba66-4ff4-9260-f7e9d642f6d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80599) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 04:35:32.964149 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.fernet_utils [None req-2cc139e7-ba66-4ff4-9260-f7e9d642f6d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80599) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 04:35:32.976203 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc139e7-ba66-4ff4-9260-f7e9d642f6d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 04:35:32.977588 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc139e7-ba66-4ff4-9260-f7e9d642f6d5 service placement] RBAC: Authorization granted {{(pid=80599) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 04:35:32.987466 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: [pid: 80599|app: 0|req: 1463/2904] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Jun 24 04:35:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 04:35:42.983254 dv-cinder-846842-3-iscsi devstack@keystone.service[80599]: DEBUG dbcounter [-] [80599] Writing DB stats keystone:SELECT=58 {{(pid=80599) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}